STAT 385 Homework Assignment 05 Due by 12:00 PM 11/16/2019 HW 5 Problems Below you will find problems for you to complete as an individual. It is fine to discuss the homework problems with classmates, but cheating is prohibited and will be harshly penalized if detected. 1. Using the ggplot function and tidyverse functionality, do the following visualizations: a. recreate your improved visualization in problem 2c of HW04 b. add a new visually appealing layer to the plot that helps clarify the plot and separately include a short description beneath the plot, such as “Fig. 1 shows…” c. recreate your improved visualization in problem 4c of HW04 d. add a new visually appealing layer to the plot that helps clarify the plot and separately include a short description beneath the plot, such as “Fig. 2 shows…” 2. Successfully import the US Natality Data (for year 2015). This single tab- delimited file link here (https://uofi.box.com/shared/static/iogdsmwxzmcqzd5jfgvdq81f44u2smc8.txt) is 1.56 GB in size. If your computer cannot handle that processing, do use the partitioned version of the data Folder link here (https://uofi.box.com/s/ksobwxvudewssucmq7fspg7trx41qce1) which are 20 comma-separated files of the same US Natality Data. Bonus (worth 10 additional points, i.e. your max HW 05 score could be 20 out of 10): do problem 2 using parallel programming ideas (particularly with mclapply) discussed in class. No outside functions/packages other than those discussed in the notes on parallel programming. 3. Using the ggplot function and tidyverse functionality, recreate or reimagine the following visualizations using the appropriate data. Be sure to use the visual design considerations from Knaflic’s Storytelling with Data. a. The image below uses the US Natlity Data. Also, explain the image with Markdown syntax. b. The image below uses the US Natlity Data. Also, explain the image with Markdown syntax (do not include the explanation within the visualization). c. The image below uses the Chicago Food Inspections Data link here (https://uofi.box.com/shared/static/5637axblfhajotail80yw7j2s4r27hxd.csv). Also, explain the image with Markdown syntax (do not include the explanation within the visualization). d. The image below uses the Chicago Food Inspections Data. Also, explain the image with Markdown syntax (do not include the explanation within the visualization). 4. Do the following: Redo problem 3 in HW03 using the mclapply function. Does parallel computing perform the tasks in parts c and e faster than the method that you used in HW03? Show your work including the runtimes for the unparallelized and parallelized versions. 5. Problem in parallel coding a. Install the conformal.glm R package which can be found at https://github.com/DEck13/conformal.glm (https://github.com/DEck13/conformal.glm). Run the following code: library(conformal.glm) set.seed(13) n # generate predictors x # set regression coefficient vector beta # generate responses from a linear regression model y # store predictors and responses as a dataframe dat # fit linear regression model model # obtain OLS estimator of beta betahat # convert predictors into a matrix Xk # extract internal model information, this is necessary for the assignment call formula family link newdata.formula # This function takes on a new (x,y) data point and reports a # value corresponding to how similar this new data point is # with the data that we generated, higher numbers are better. # The goal is to use this function to get a range of new y # values that agrees with our generated data at each x value in # our generated data set. density_score rank(phatxy(ynew = ynew, xnew = xnew, Yk = y, Xk = Xk, xnew.modmat = xnew, data = dat, formula = formula, family = family, link = link))[n+1] } # We try this out on the first x value in our generated data set. # In order to do this we write two line searches xnew # start line searches at the predicted response value # corresponding to xnew ystart score # line search 1: line search that estimates the largest y # value corresponding to the first x value that agrees with # our generated data while(score > 13){ yupr score } # line search 2: line search that estimates the smallest y # value corresponding to the first x value that agrees with # our generated data score while(score > 13){ ylwr score } b. Write a function which runs the two line searches in part a for the jth generated predictor value. c. Use mclapply to run the function you wrote in part b, set the mc.cores argument to the output of a call to the detectCores() function. Save the output and record the time that it took to perform these calculations Note that setting the number of cores in mclapply to the number of available cores could grind your computer to a halt if you are using your computer for other tasks. d. Redo the calculation in part c using lapply and record the time it took to run this job. Which method was faster? e. Using ggplot, plot the original data and depict lines of the lower and upper boundaries that you computed in part c.
