Working with big data: tidying, wrangling, visualising, modelling repoducibly and collaboratively

Presenter: Dianne Cook, a Professor of Statistics at Monash University in Melbourne, Australia, is a global leader in data visualisation. She has delivered over 100 invited talks internationally and published extensively on various aspects of data visualisation. Dr. Cook is a Fellow of the American Statistical Association, an elected member of the International Statistical Institute, past editor of the Journal of Computational and Graphical Statistics, and the R Journal. She has served as a Board Member of the R Foundation and is currently the co-chair of the Statistical Computing and Visualisation Section of the Statistical Society of Australia.

The workshop will use a hands-on teaching methodology that combines short lectures with longer practice sessions. Whether you decide to follow along or not, you will be able to reproduce all the slides and examples, later.

We assume some familiarity with R, and will teach using R and Rstudio.

Outline

Session	topic	description	source
1	Introduction, tidy data analysis and workflow	This session covers key principles for handling large, complex datasets and tools, e.g. identifying data as in tidy format, or not, data cleaning and integration and how to build a robust workflow, that enables collaborative data analysis and reproducible reporting.	R code, quarto
2	Data wrangling, and visualisation	From data in tidy form to elegant graphics, using a grammar to make good visualisations, and identifying patterns and communicating evidence.	R code, quarto
3	Statistical modeling and machine learning	Extending the notion of tidy data to models provides a cleaner way to think about the process of prediction and interpretation.	R code, quarto
4	Application 1	Working with Indonesian economic data.	R code, quarto
5	Interactive and dynamic graphics	Here we learn how to visualise data beyond two dimensions, to assess relationships between many variables.	R code, quarto
6	Application 2	Working with interactive plots for spatial and temporal data about Indonesia.	R code, quarto

And setup code files: libraries.R, chunk_options_and_themes.R.

To work on application 1

Make sure to have a reasonably recent version of R and RStudio.

Install these R packages

install.packages(c("ggplot2", 
                "tidyr", 
                "dplyr", 
                "stringr", 
                "tidymodels",
                "ggbeeswarm", 
                "nullabor", 
                "ranger",
                "patchwork",
                "naniar",
                "visdat",
                "lubridate",
                "ggthemes",
                "conflicted"), 
                dependencies=c("Depends", "Imports"))

Download the application1.zip file, unzip to your working directory.

To produce reproducible documents

Install quarto by following the instructions here.

To work on application 2

Install these additional packages

install.packages(c("readr", 
                "plotly",
                "sf",
                "crosstalk",
                "cartogram"
), 
                dependencies=c("Depends", "Imports"))

Download the application2.zip and unzip in the same folder as for application 1.

Resources

The end of each slide set has references and links to more information.

These materials are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.