Business Analytics Using Python

Business Analytics Using Python

Abstract

This paper aims at using a data set to conduct hypothesis testing. The study will be introduced by explaining the importance of the hypothesis and the application of hypothesis testing. The hypothesis is formulated in the introduction section. Under the body section, the test statistics are introduced and explained in depth. The results obtained are described under the findings and conclusion section. The results are explained, and the hypothesis is tested under this section. The results obtained are connected with the original ideas. The codes that were used to do the testing will be pasted under the appendix section. Finally, the relevant references are placed and the references section.

1.      Introduction

The paper aims at using the concept of data to make some business decisions. It is widely known as business intelligence. This concept is commonly used in the current market. Typically, human beings have been using their methods of making the decision, and some have worked, and others have failed terribly. Then humans discovered machines to help them with their i.e., to simplify their work. The machines have dominated everywhere from the business sectors, technology, education, health, etc. In business, the CEO focuses mostly on satisfying their clients and making sure that they increase their sales. The artificial intelligence has been used to help human beings make the decision. In that, the machines are frequently trained until they master the concept, and by doing this, they can make their own decision without being controlled by the human. This is the reason why machines can sense the presence of danger when there is smoke in certain places. The same concept of using machine learning in making the decision has been witnessed in new businesses. Different companies have been collecting data from their clients and use the same data to know and predict the behavior of the clients. The concept of machine learning, which goes hand in hand with data prediction, has been used by many businesses to make predictions of the companies and also in decision making. There is a frequency of using technology by industry to cope up with the competitive market. It is very crucial for companies to understand their clients and to know their behavior in a way that they can control them while at the same time, they are retaining their customers.

In this report, a real-time data set is going to be used to make some predictions. The data sets that have been provided are about personal earning in the year 2011 and 2012 and the year 2012 and 2013. The data contains information about individuals making such the earning of the male and female, the earning of self-employed and those who are not self-employed. The data contains 689080 observations and 47 variables. Four hypotheses need to be created to be answered using the data. Therefore, four questions need to be created using focusing on the data. The four questions that will be explained using the data are:

1. Is there any relationship between Profit and Pension and the total income earned by a person
2. Does the Total Income differ by gender?
3. Does Total income differ with the age groups?
4. Does total investment income differ between the self-employed cases?

With the above information, the hypotheses can be formulated to answer the above questions. The formulated hypotheses will be validated by using the results obtained from the analysis.

Hypothesis

The following are null hypotheses formulated to answer the questions

H0: There is no significant relationship between Profit and Pension with the total income earned by the individuals

H1: The total income earned by the female is equal to the total income earned by the male

H2: The total income earned is the same in all the age groups

H3: The total income earned by those who are self-employed is similar to the total income earned by those who are not self-employed.

2.      Body Sections

The hypothesis is statements of the assumption that are used to interpret certain events by guiding what needs to be investigated. Statistics, it formulated before conducting a test. Sometimes, an analysis may want to get straight information about a particular phenomenon, and hypothesis testing is always the best approach that can be used in such a situation. To be able to get specific information, one needs to come up with specific ideas about the final results. The idea might either be wrong or right.

In the same way, the hypothesis is divided into two, the first type of hypothesis is the null hypothesis, and the other is the alternate hypothesis. If one of the hypotheses is rejected, the other is accepted and vice versa. In this research, we will focus on individual earning between the years 2011 and 2012. The income of an individual is determined by many factors, including the type of jobs done by the individuals, whether the individual is running a business, the profits obtained by the individual doing business, and many more factors. The research will be conducted using a different statistical test. The statistical tests that will be conducted will be determined by the type of research questions. Some of the tests determine even the strength of the test.

The hypothesis test is determined using the p-value. The standard p-value that is used to test the hypothesis is both 0.05 and 0.01. These are the standard p-value that is used to test the hypotheses. However, any p-value can be used by the researcher when examining the data. The research questions will be answered using the validated hypothesis i.e., the research question will either be explained using the null hypothesis or the alternative hypothesis. The significance of the test is what determines what to consider. There are several types of tests that will be conducted to answer the research questions. The statistical test that we will use to answer the questions is the regression analysis, the independent t-test, and the ANOVA test.

Linear regression analysis

Regression analysis is a test that is used to test the relationship between the independent and dependent variables. It is related in correlation in that they both determine the relationship between the dependent and independent variables. However, the association is only limited to determine the relationship between the dependent and independent variables, but the regression analysis is itself a model. It identifies the coefficients, the significance of the link, and the fitness of the relationship between the dependent and independent variables. There are two types of regression analysis. The first type is the simple linear regression, and the second type is the multiple linear regressions. The simple linear regression occurs when there is one dependent variable and one independent variable. The multiple linear regressions happen when there is more than one independent variable. In our case, we will build a multiple regression model since there are two independent variables.

The regression equation is used to identify which variables have an impact on the topic of interest. The regression model determines which factors matter most and the factors that should be ignored. There two variables that need to be comprehended are;

• Dependent variable: This is the factor that is being predicted
• Independent variables: These are the factors that are assumed to have an impact on the topic of interest i.e., the dependent variable

In our application on the first hypothesis, the dependent variables will be the total income, while the independent variables will be the profits and the pension. To investigate the relationship between the total income and profits and the relationship between the total income and the pension, we will first create a scatter chart to check their relationship graphically. After the scatter plot has been created, one can see the relationship between the total income and profits and the relationship between the total income and the pension. One can see the direction and strength of the relationship between the above.

The regression formula is summarized as

y = MX + b

Y stands for the dependent variable. M stands for the slope of the regression. X stands for the independent variable in the regression, and b stands for the constant of the regression equation, also known as the y-intercept.

The significance of the regression model will determine whether the null hypothesis is rejected, or it is accepted. It will also determine the variable, which is the best predictor of the total income. The multiple R will evaluate the relationship between the pension and profits with the total income (Seber, & Lee, 2012; Chatterjee & Hadi, 2015; Montgomery, Peck & Vining, 2012; Cameron & Trivedi 2013). The value of the R squared will be used to determine the fitness of the regression model. The amount of the r square is used to check the variability of the dependent variable using the independent variables. An r square value between 0 to 0.3 is considered weak fitness. The r squared value between 0.3 to 0.6 is considered as the medium fitness, while the r square value of between 0.6 to 1 is regarded as the strong fitness.

T-test

The t-test is used to test the significance between groups i.e., and it is used to determine the mean difference between groups, usually one or two. It focuses on the averages of the groups. A t-test is used as a testing tool for the hypothesis, which tests the assumptions of a specific population.

The t-test works when one wants to compare the average values of the two data sets that are from one population. In the second hypothesis, we will take a sample of total income for the male and the total income for the females. These are two data sets but in the same population. There is no expectation that the total income for the female and the total income for the male will have the same means and the standard deviation. The same concept applies to the fourth hypothesis, where the total income for the self-employed and the total income for those who are not self-employed cannot have equal mean and standard deviation. The t-test is usually the sample taken from each of the two sets and establishes a problem statement by formulating a null hypothesis that the means of the two sets are equal. Defined values are calculated of which are to be compared with the standard values. Later, the null hypothesis is accepted or rejected accordingly.

Some assumptions are made before conducting the t-test. The first assumption is that the scale of measurement of the variables should either be continuous or ordinal. The second assumption is made concerning the sampling of the population. The assumption is made that the sample population is selected randomly. This aims to avoid bias results when the sample is not chosen randomly. The third assumption is built on the dependent variable. The distribution of the dependent variable should be normal i.e., normally distributed. The fourth assumption is made on the size of the sample. The assumption assumes that the size of the sample population is large. This helps produce a normally distributed population. The fifth assumption assumes the homogeneity of the variance i.e., equal variance (Kim, 2015; De Winter, 2013).

When calculating the t-test, the mean and the standard deviation plays a significant role in. Many types of t-test can be used to determine the differences in the means of groups. These groups are the independent sample t-test, paired t-test, one-sample t-test, and two-sample t-test. In our research question, two hypotheses will be tested using the t-test. We will test whether the total income earned by the female is equal to the total income earned by the male. The second hypothesis that will be tested that will be answered using the t-test is to determine whether the total income for the self-employed is equal to the total income for those who are not self-employed.

ANOVA

ANOVA is an inferential statistical technique that is used to the means of two or more groups. It is slightly similar to the t-test; the only difference is that it can test the means of more than two groups, unlike t-test, which can’t test the difference between more than two groups. Similar to the t-test, it also has some assumptions that are made when conducting it. The first assumption that is made that the population must be close to or normal distributed. The second assumption that is made is that the sample must be independent of each other. The third assumption made is the homogeneity of the variances i.e., the variances are equal. The last assumption that is made is that the sample must have similar sample sizes. The third hypothesis will be tested using ANOVA. The age group has seven groups, and this makes an ANOVA the best test to be carried out to test the hypothesis (Wetzels, Grasman & Wagenmakers, 2012; Jan & Shieh, 2014).

3.      Finding and Conclusion

1. Descriptive Statistics

The data was subsetted into seven relevant columns. Out of the seven variables, three are categorical variables while the rest are the numerical variables

Table 1.1: The descriptive statistics of the Numerical variables

Table 1 shows that the average pension of the individual in the year 2011 to 2012 was € 13,125. The minimum and maximum amount of pension issued in the year 2011 to 2012 was € 5 and € 1000000, respectively. The deviation of the pension given in the year 2011 to 2012 from its actual means was € 26056. The average profit obtained by the individuals in the year 2011 to 2012 was € 24042.2. The minimum and maximum amount of profits received by the individuals in the year 2011 to 2012 was € 5 and € 2,500,000, respectively. The deviation of the profits received by the individuals in the year 2011 to 2012 from its actual means was € 60250.2. The average total income obtained by the individuals in the year 2011 to 2012 was € 50236.1. The minimum and maximum amount of total income received by the individuals in the year 2011 to 2012 was € 210 and € 3,454,890, respectively. The deviation of the total income received by the individuals in the year 2011 to 2012 from its actual means was € 97243.38. The average total investment income obtained by the individuals in the year 2011 to 2012 was € 5838. The minimum and maximum amount of total investment income received by the individuals in the year 2011 to 2012 was € 0 and € 1,876,080, respectively. The deviation of the total investment income received by the individuals in the year 2011 to 2012 from its actual means was € 31,562.87.

SEX

Out of the total population 414,943 (60.22 %) was Male while 274,136 (39.78 %) were female. The same approach can be summarized graphically, as shown below.

Figure 1.1: The distribution of the gender

Age group

Out of the total population, 62,560 (9.08 %) were below 25 years old. 113,430 (16.64 %) were 25 to 34 years old. 135,860 (19.72 %) were 35 to 44 years old. 143,838 (20.87 %) were 45 to 54 years old. 104,556 (15.17 %) were 55 to 64 years old. 76,527 (11.11 %) were 65 to 74 years old, and finally, 52,309 (7.59 %) were 75 years and over. The same concept can be represented using the bar chart, as shown below.

Figure 1.2: Distribution of the Age group

The Employment Cases

Out of the total population, 564,406 (81.91 %) were not self-employed while 124,612 (18.08 %) were self-employed. This information can be represented using a bar chart as shown below

Figure 1.3: The bar chart of the Employment Cases

Testing hypothesis 1

The relationship between Total income and the profits is 56.43, and the relationship between the total revenue and the pension is 59.21 %. The same can be viewed in using scatter plot as shown below

Figure 1.4: Total Income VS Profits

Fig 1.5: Total Income Vs. Pension

The regression model obtained after the analysis can be summarized as

Total Income = 1.045 (PROFITS) +1.55 (PENSION) + 4765.56

This means that an increase in a unit profit increases the total income by 1.045. A unit increases in pension increases the total income by 1.55. However, the total income of £ 4765.56 is not affected by both Profits and pension. The coefficient of determination was obtained to be 0.6892. This means that the model explained 68.92 % variation of the Total income. With this information, we can conclude that there is a relationship between the pension and profits with the total income. It is evident that those who receive high profits also have a higher total income.

Similarly, pension increase the total income of an individual (Gill, Biger & Mathur, 2010; Weiss, 2010; Almazari, 2014; Dharmapala & Riedel, 2013)

Testing Hypothesis 2

Below is the output obtained from the independent t-test

Table 1.2: The independent sample t-test

The obtained p-value is 0.000. The obtained p-value is less than 0.05 i.e., P (0.000 < 0.05); thus, the null hypothesis is rejected, and we conclude that the total income earned by the female is different from the total revenue earned by the male.

Table 1.3: Descriptive Statistics

The above descriptive statistics give further information that the total income for the males was higher than those of the females i.e., the mean total income of the males is higher than that of females. It is evident even from the society that there is a gender wage gap where you find the males are being paid a higher amount compared to the females. The reason for such is that the mean tends to focus on the technical jobs which are being paid. Another reason might be because of the discrimination against the gender (Angelov, Johansson & Lindahl, 2016; Graf, Brown & Patten, 2018; Card, Cardoso & Kline, 2015; Manning & Saidi, 2010; Lips, 2013; De la Rica, Dolado & Vegas, 2010).

Testing the third hypothesis

The p-value obtained from the One-Way ANOVA is 0.00034. The obtained p-value is less than 0.05 i.e., P (0.00034 <0.05); thus, the null hypothesis is rejected, and therefore we conclude that the total income is different among the different age groups. Age is a very crucial factor when determining the salary of individuals. The age of a person sometimes determines the years of experience an individual has. People with many years of experience receive a higher salary compared to those that have a few years of experience. For this reason, their total income can’t be equal (Bischoff & Reardon, 2014; Laghari & Connelly, 2012).

Testing the fourth hypothesis

Table 1.4: Independent sample t-test

Table 1.4 shows the t-test statistics output obtained while trying to determine the difference in the means of the total investment income of those who are self-employed and those who are not self-employed. The obtained p-value is less than 0.005 i.e., P (0.000<0.05); thus, we conclude that the total investment income for those who are self-employed is different from the income of those who are not self-employed.

Table 1.5: Descriptive Statistics

The table above shows that those who were self-employed had a higher total investment income compared to those who were not independent. The mean of the total investment income of those who were self-employed was £ 4692 while the total income for those who were not self-employed was £ 4091. The reason why the total investing income is higher among the self-employed is that some of those who were not self-employed might be unemployed. Again, most of the self-employed practice planning and thus invest often. This might be another reason why those who are self-employed have a higher total investing income (Schoar, 2010; Kuvaas & Dysvik, 2010; Dohmen et al. 2011).

4.      Appendix

5.      References

Almazari, A. A. (2014). Impact of internal factors on bank profitability: Comparative study between Saudi Arabia and Jordan. Journal of Applied finance and banking, 4(1), 125.

Angelov, N., Johansson, P., & Lindahl, E. (2016). Parenthood and the gender gap in pay. Journal of Labor Economics, 34(3), 545-579.

Bischoff, K., & Reardon, S. F. (2014). Residential segregation by income, 1970–2009. Diversity and disparities: America enters a new century, 43.

Cameron, A. C., & Trivedi, P. K. (2013). Regression analysis of count data (Vol. 53). Cambridge university press.

Card, D., Cardoso, A. R., & Kline, P. (2015). Bargaining, sorting, and the gender wage gap: Quantifying the impact of firms on the relative pay of women. The Quarterly Journal of Economics, 131(2), 633-686.

Chatterjee, S., & Hadi, A. S. (2015). Regression analysis by example. John Wiley & Sons.

De la Rica, S., Dolado, J. J., & Vegas, R. (2010). Performance pay and the gender wage gap: evidence from Spain.

De Winter, J. C. (2013). Using the Student’s t-test with extremely small sample sizes. Practical Assessment, Research & Evaluation, 18(10).

Dharmapala, D., & Riedel, N. (2013). Earnings shocks and tax-motivated income-shifting: Evidence from European multinationals. Journal of Public Economics, 97, 95-107.

Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., & Wagner, G. G. (2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9(3), 522-550.

Gill, A., Biger, N., & Mathur, N. (2010). The relationship between working capital management and profitability: Evidence from the United States. Business and economics journal, 10(1), 1-9.

Graf, N., Brown, A., & Patten, E. (2018). The narrowing, but persistent, gender gap in pay. Pew Research Center, April, 9.

Jan, S. L., & Shieh, G. (2014). Sample size determinations for Welch’s test in one‐way heteroscedastic ANOVA. British Journal of Mathematical and Statistical Psychology, 67(1), 72-93.

Kim, T. K. (2015). T test as a parametric statistic. Korean journal of anesthesiology, 68(6), 540.

Kuvaas, B., & Dysvik, A. (2010). Exploring alternative relationships between perceived investment in employee development, perceived supervisor support and employee outcomes. Human Resource Management Journal, 20(2), 138-156.

Laghari, K. U. R., & Connelly, K. (2012). Toward total quality of experience: A QoE model in a communication ecosystem. IEEE Communications Magazine, 50(4), 58-65.

Lips, H. M. (2013). The gender pay gap: Challenging the rationalizations. Perceived equity, discrimination, and the limits of human capital models. Sex Roles, 68(3-4), 169-185.

Manning, A., & Saidi, F. (2010). Understanding the gender pay gap: what’s competition got to do with it?. ILR Review, 63(4), 681-698.

Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis (Vol. 821). John Wiley & Sons.

Schoar, A. (2010). The divide between subsistence and transformational entrepreneurship. Innovation policy and the economy, 10(1), 57-81.

Seber, G. A., & Lee, A. J. (2012). Linear regression analysis (Vol. 329). John Wiley & Sons.

Weiss, D. (2010). Cost behavior and analysts’ earnings forecasts. The Accounting Review, 85(4), 1441-1471.

Wetzels, R., Grasman, R. P., & Wagenmakers, E. J. (2012). A default Bayesian hypothesis test for ANOVA designs. The American Statistician, 66(2), 104-111.