Lecture 2

class: center, middle, inverse, title-slide

# Lecture 2
### Jung-Jin Lee
### Jan 15, 2019

---

## Type of data

- Categorical Data - consist of qualitative values
  - Nominal values: gender, coat colors
  - Ordinal values: body conditions, age groups
  
- Numerical Data - consist of quantitative values
  - Discrete values: litter size, parity
  - Continuous values: height, weight, temperature
  
---

## Explore data

```r
library(tidyverse)
df <- diamonds ## diamonds comes with ggplot2
dim(df)
```

```
## [1] 53940    10
```

```r
head(df)
```

```
## # A tibble: 6 x 10
##   carat cut       color clarity depth table price     x     y     z
##   <dbl> <ord>     <ord> <ord>   <dbl> <dbl> <int> <dbl> <dbl> <dbl>
## 1 0.23  Ideal     E     SI2      61.5    55   326  3.95  3.98  2.43
## 2 0.21  Premium   E     SI1      59.8    61   326  3.89  3.84  2.31
## 3 0.23  Good      E     VS1      56.9    65   327  4.05  4.07  2.31
## 4 0.290 Premium   I     VS2      62.4    58   334  4.2   4.23  2.63
## 5 0.31  Good      J     SI2      63.3    58   335  4.34  4.35  2.75
## 6 0.24  Very Good J     VVS2     62.8    57   336  3.94  3.96  2.48
```

---

## Explore data, continued

```r
names(df)
```

```
##  [1] "carat"   "cut"     "color"   "clarity" "depth"   "table"   "price"  
##  [8] "x"       "y"       "z"
```

---

## Explore data, continued

How many distinct values are there in the variable `cut`?

```r
DiaCut <- df$cut
unique(DiaCut)
```

```
## [1] Ideal     Premium   Good      Very Good Fair     
## Levels: Fair < Good < Very Good < Premium < Ideal
```

What is the frequency of each value?

```r
table(DiaCut)
```

```
## DiaCut
##      Fair      Good Very Good   Premium     Ideal 
##      1610      4906     12082     13791     21551
```

---

## Bar graph

One way to visualize categorical data is to use a bar graph

```r
g <- ggplot(data = df, aes(x = cut)) + 
  geom_bar() 
print(g)
```

---

## Exercise

- How many distinct values are there in the variable `color`?
- What is the frequency of each value in `color`?
- Draw a bar graph showing the frequency

```r
unique(df$color)
```

```
## [1] E I J H F G D
## Levels: D < E < F < G < H < I < J
```

```r
table(df$color)
```

```
## 
##     D     E     F     G     H     I     J 
##  6775  9797  9542 11292  8304  5422  2808
```

---

## Exercise, continued

```r
ggplot(df, aes(color)) + ## can omit 'data = ' and 'x = '
  geom_bar()
```

---

## Two-way table

Frequency of color for each cut

```r
table(df$cut, df$color)
```

```
##            
##                D    E    F    G    H    I    J
##   Fair       163  224  312  314  303  175  119
##   Good       662  933  909  871  702  522  307
##   Very Good 1513 2400 2164 2299 1824 1204  678
##   Premium   1603 2337 2331 2924 2360 1428  808
##   Ideal     2834 3903 3826 4884 3115 2093  896
```

---

## Graphs using facets

```r
ggplot(df, aes(color)) + ## can omit 'data = ' and 'x = '
  geom_bar() +
* facet_wrap(~cut)
```

---

## Colored bar graphs

```r
*ggplot(df, aes(color, fill = clarity)) +
  geom_bar() +
  facet_wrap(~cut) 
```

---

## Exercise

Produce the following plots:

---

## Description of numerical data

What is the distribution of `carat` like?

```r
ggplot(df, aes(carat)) +
  geom_histogram()
```

---

## Location and spread of numerical data

- Measure of location: sample mean and median
- Measure of spread: sample variance and standard deviation

---

## Example: find the sample mean

The sample mean is given by sum of samples divided by the number of samples:
`$$\bar{x} = \frac{x_1+x_2+\cdots+x_n}{n}$$`

```r
d <- read.table(file = "heights.txt", header = TRUE, sep = " ")
mom <- d$Mheight
s <- sum(mom); n <- length(mom)
mom_height_mean <- s/n
print(mom_height_mean)
```

```
## [1] 62.4528
```

R has a built in function to compute the mean:

```r
mean(mom) # mean() gives the mean of a vector
```

```
## [1] 62.4528
```

---

## Sample median

The median is the value separating the higher half from the lower half of numerical data. R has a built in function to compute the median.

```r
median(mom)
```

```
## [1] 62.4
```

In general, one can compute quantiles (cut points dividing range of numerical data).

```r
quantile(mom)
```

```
##   0%  25%  50%  75% 100% 
## 55.4 60.8 62.4 63.9 70.8
```

The median is the same as the 50% quantile.

---

## Example

**Exercise**: Find the sample mean and median of `carat`.

```r
carat <- df$carat
mean(carat); median(carat)
```

```
## [1] 0.7979397
```

```
## [1] 0.7
```

**Exercise**: Find the 90% quantile `carat`.

```r
quantile(carat)
```

```
##   0%  25%  50%  75% 100% 
## 0.20 0.40 0.70 1.04 5.01
```

```r
quantile(carat, probs = 0.9)
```

```
##  90% 
## 1.51
```

---

## Sample variance

The sample variance measures spread of given data:
`$$s^2 =\frac{(x_1-\bar{x})^2+(x_2-\bar{x})^2+\cdots+(x_n-\bar{x})^2}{n-1}.$$`
Note that the denominator is `$n-1$`, not `$n$`.

R has a built in function to compute the sample variance:

```r
var(mom) # var() gives the sample variance of a vector
```

```
## [1] 5.546511
```

One could also compute the sample variance using the definition:

```r
v1 <- mom-mean(mom); v2 <- v1^2
sum(v2)/(length(mom)-1)
```

```
## [1] 5.546511
```

---

## Sample standard deviation

The sample standard deviation also measures spread of given data -- it is simply the square root of the sample variance:
`$$s = \sqrt{s^2} = \sqrt{\frac{(x_1-\bar{x})^2+(x_2-\bar{x})^2+\cdots+(x_n-\bar{x})^2}{n-1}}.$$`

R has a built in function to compute the sample standard deviation:

```r
sd(mom) # sd() gives the sample standard deviation of a vector
```

```
## [1] 2.355103
```

---

## Exercise

- Produce the histogram of daughter's heights from the dataset `heights.txt`.

- Get mean/median/sample variance of daughter's heights.

---

## Exercise, continued

```r
ggplot(d, aes(Dheight)) + geom_histogram()
```

```r
D_height <- d$Dheight
c(mean(D_height), median(D_height), var(D_height))
```

```
## [1] 63.751055 63.600000  6.760274
```