Lecture 5

class: center, middle, inverse, title-slide

# Lecture 5
## Introduction to statistical tests
### Jung-Jin Lee
### Feb 4, 2020

---

## One-sample _t_-test

Suppose that a sample of size `$n$`, say `$X_1,X_2,\ldots,X_n$`, is taken from a population that follows `$N(\mu,\sigma^2)$` (this condition can be dropped if `$n$` is large enough). Then `$\frac{\bar{X}-\mu}{S/\sqrt{n}}$`  follows a `$t$`-distribution with `$n-1$` degrees of freedom, where
- `$\bar{X}$`: sample mean 
- `$S$`: sample standard deviation

---

## One-sample _t_-test, example

```r
ht <- read.table(file = "heights.txt", header = T, sep = " ")
ggplot(ht, aes(Mheight)) + 
  geom_histogram()
```

---

## One-sample _t_-test, example

- Assume UK mom's heights follow a normal distribution `$N(\mu, \sigma^2)$`.

- A hypothesis is given: population (all UK mom's heights) mean `$\mu$` is 62.2. This is called the _null hypothesis_ and usually denoted by `$H_0$`.

- Does our data `ht` seem to support the null hypothesis? Or does it provide evidence against the null hypothesis?

- This question can be answered using one-sample _t_-test.

---

## One-sample _t_-test, example

```r
mu <- 62.2 # null hypothesis
x <- ht$Mheight
xbar <- mean(x); S <- sd(x); n <- length(x)
t <- (xbar - mu) / (S/sqrt(n)) 
print(xbar)
```

```
## [1] 62.4528
```

```r
print(S)
```

```
## [1] 2.355103
```

```r
print(n)
```

```
## [1] 1375
```

```r
print(t)
```

```
## [1] 3.980324
```

---

## One-sample _t_-test, example

Getting 3.98 seems to be a rare event. How rare is it?

---

## One-sample _t_-test, example

`$p$`-value: probability of getting a result that is as extreme as or more extreme than current observation.

```r
2*(1- pt(t, df = n-1)) # p-value
```

```
## [1] 7.241748e-05
```

---

## One-sample _t_-test, example

- Under the hypothesis that `$\mu=62.2$`, our current observation seems to be a really rare event.

- How rare? The obtained `$p$`-value is less than 0.05. This is called a _significance level_.

- Maybe this is due to _unreasonable_ hypothesis.

- So it is natural to _reject_ the null hypothesis.

- We conclude that the population mean is not equal to 62.2 (alternative hypothesis).

---

## One-sample _t_-test, example

```r
t.test(x, mu = 62.2)
```

```
## 
## 	One Sample t-test
## 
## data:  x
## t = 3.9803, df = 1374, p-value = 7.242e-05
## alternative hypothesis: true mean is not equal to 62.2
## 95 percent confidence interval:
##  62.32821 62.57739
## sample estimates:
## mean of x 
##   62.4528
```

---

## One-sample _t_-test, exercise

Assume that UK young women's heights follow a normal distribution `$N(\mu, \sigma^2)$`. Using one-sample _t_-test and data `Dheight`, test the null hypothesis `$H_0:\mu = 63.8$`.

.pull-left[
<img src="Drexel_2020_Lecture_5_files/figure-html/unnamed-chunk-8-1.png" style="display: block; margin: auto;" />
]

.pull-right[
<img src="Drexel_2020_Lecture_5_files/figure-html/unnamed-chunk-9-1.png" style="display: block; margin: auto;" />
]

---

## One-sample _t_-test, exercise

```r
t.test(ht$Dheight, mu = 63.8)
```

```
## 
## 	One Sample t-test
## 
## data:  ht$Dheight
## t = -0.69804, df = 1374, p-value = 0.4853
## alternative hypothesis: true mean is not equal to 63.8
## 95 percent confidence interval:
##  63.6135 63.8886
## sample estimates:
## mean of x 
##  63.75105
```

Based on the output above, we do _not_ reject the null hypothesis.

---

## Exercise

Recall `iris` data set. Based on the `Sepal.Length` data of 50 `setosa` species, can you say that the population mean of `Sepal.Length` is equal to 5.1? Use significance level of 0.05.

.pull-left[

```r
ds <- iris %>%
  filter(Species == "setosa")
```

```r
ggplot(ds, aes(Sepal.Length)) + 
  geom_histogram(bins = 10)
```
]

.pull-right[
<img src="Drexel_2020_Lecture_5_files/figure-html/unnamed-chunk-13-1.png" style="display: block; margin: auto;" />
]

---

## Exercise, continued

```r
t.test(ds$Sepal.Length, mu = 5.1)
```

```
## 
## 	One Sample t-test
## 
## data:  ds$Sepal.Length
## t = -1.8857, df = 49, p-value = 0.06527
## alternative hypothesis: true mean is not equal to 5.1
## 95 percent confidence interval:
##  4.905824 5.106176
## sample estimates:
## mean of x 
##     5.006
```

---

## Exercise

Download a tab-delimited file [crabs.tsv](crabs.tsv). It consists of 173 observations and 5 variables.

```r
crabs <- read.table("crabs.tsv", header = T, sep = "\t") 
dim(crabs)
```

```
## [1] 173   5
```

```r
head(crabs)
```

```
##   color spine width satell weight
## 1     3     3  28.3      8   3050
## 2     4     3  22.5      0   1550
## 3     2     1  26.0      9   2300
## 4     4     3  24.8      0   2100
## 5     4     3  26.0      4   2600
## 6     3     3  23.8      0   2100
```

---

## Exercise, continued

A nesting female horseshoe crab may have male crabs residing nearby, called satellites, besides the male crab residing in her nest. Brockmann (1996) investigated factors (including the female crab's color, spine condition, weight, and carapace width) which may influence the presence/absence of satellite males.

---

## Exercise, continued

- color: the female crab's color, coded 1: light; 2: medium light; 3: medium; 4: medium dark; 5: dark. Not all of these colors appear.

- spine: the female crab's spine condition, coded 1: both good; 2: one worn or broken; 3: both worn or broken.

- width: the female crab's carapace width in cm.

- satell: the number of satellite males.

- weight: the female crab's weight in grams.

`color` and `spine` are not really numeric. To treat them as non-numeric we use `factor()`. Also create `id` column for each observation.

```r
crabs <- crabs %>%
  mutate(color = factor(color)) %>%
  mutate(spine = factor(spine)) %>%
  mutate(id = paste0("obs", 1:nrow(crabs)))
```

---

## Exercise, continued

- Create the following plots: histograms of `width` faceted by `spine` conditions and box plots of `weights` by `color`. Set the number of bins in each histogram to 10. Identify the two outliers in the box plots.

.pull-left[
<img src="Drexel_2020_Lecture_5_files/figure-html/unnamed-chunk-17-1.png" style="display: block; margin: auto;" />
]

.pull-left[
<img src="Drexel_2020_Lecture_5_files/figure-html/unnamed-chunk-18-1.png" style="display: block; margin: auto;" />
]

---

## Exercise, continued

- Assume that the `width` of horse shoe crabs follows a normal distribution of `$N(\mu, \sigma^2)$`. Based on the 173 observations, test the hypothesis `$H_0: \mu = 26$`. What is your conclusion? Report the sample mean and 95% confidence interval of `$\mu$`.

- Assume that the `weight` of horse shoe crabs follows a normal distribution of `$N(\mu, \sigma^2)$`. Based on the 173 observations, test the hypothesis `$H_0: \mu = 2300$`. What is your conclusion? Report the sample mean and 95% confidence interval of `$\mu$`.