Disentangling Data Science
Category: visualization
Joel Simon is the genius behind an experimental project exploring optimized school blueprints. Joel used graph-contraction and ant-colony pathing algorithms as growth processes, which could generate elementary school designs optimized for all kinds of characteristics: walking time, hallway usage, outdoor views, and escape routes just to name a few.
Aleszu Bajak at Storybench.org published a great demonstration of the power of text mining. He used the R tidytext package to analyse 150,000 wine reviews which Zach Thoutt had scraped from Wine Enthusiast in November of 2017.
Aleszu started his analysis on only the French wines, with a simple word count per region:
ggplot2 and the maps package to demonstrate which French wine regions are generally more costly.
Nothing beats a aesthetically-pleasing data visualization in the form of a map (see evidence here, here, here, or here).
Moreover, we’ve already witnessed some great R tutorials by Ilya Kashnitsky before (see Animated Snow in R).
These two come together in Ilya’s recent post on subplots in ggplot2 maps, with which he completely amazed me. The creation process is actually easier than the end result makes it look: make several visualizations and add them as ggplot2::annotation_custom() to your main ggplot2 map — the same as if you are adding a logo to your plot. Enjoy:
Here you can find Ilya’s original blog and the associated R script.
Below are a dozen of very specific R tips and tricks. Some are valuable, useful, or boost your productivity. Others are just geeky funny.
More general helpful R packages and resources can be found in this list.
If you have additions, please comment below or contact me!
Completely new to R? → Start here!
hereALT + left mousebutton to select and write on multiple lines simultaneously. ALT + - to insert a <- operatorCTRL + SHIFT + M to insert a %>% operatorCTRL + SHIFT + F to search all files in the directory or projectCTRL + UP to access navigate your console historyCMD + ALT + SHIFT + M to rename variable within scope: to rename all/multiple occurrences of a variable in a scriptTAB inside “” (quotation marks / an empty string) to select a filename from your current directory, or to autocomplete a filename you started typingMany more shortkeys available here online, and in your RStudio under Tools → Keyboard Shortcuts Help.
datapastacorrr to simplify working with correlationsprettycode to print highighted console outputsnake_case formatmagick::image_read('screenshot:')RVerbalExpression package
Disclaimer: This page contains one or more links to Amazon.
Any purchases made through those links provide us with a small commission that helps to host this blog.
str() – explore structure of R objecttrimws() – trim trailing and/or leading whitespacesdput() – dump an R object in form of R codecut()– categorize values into intervalsintersect() – returns similar elements in two vectorsunion() – find intersecting items in two vectorssetdiff() – returns different elements in two vectorsinteraction() – computes a factor which represents the interaction of the given factorsformatC()can be used to round numbers and force trailing zero’sformatC() and sprintf() can be used to add leading/trailing charactersexpand.grid() – create a data frame from all combinations of the supplied vectors or factorsseq_along(myvec) – generates a vector of 1:length(myvec)switch() can replace elaborate ifelse statements (see also)match.arg() can check for arguments and values%||%) returns the first value that is not NULLoptions() in your script: options(stringsAsFactors = FALSE)options(width = 60) to change the default width of console outputoptions(max.print = 100) to change the default number of values printed in the console[text](url){target = "_blank} in markdown#<< to highlight codexaringan slide upon click (or Enter)knitr::purl(input, output, documentation = 2)CTRL + SHIFT + 1:4 to zoom in on any single of your RStudio panels. Use ALT + CTRL + SHIFT + 0 to zoom back out.knitr::read_chunk("your_script_name.R") can be used to source in scripts that reside outside your current markdown filegganimate package and "header-includes: - \usepackage{animate} in your YAML preambleDT::datatable(mydf, filter = 'top') readr::parse_number extracts the numbers from raw / scraped textstringr::str_pad can be used to add leading or trailing characters (like zero’s) dplyr tricksdplyr::case_when replaces elaborate ifelse statements (Video)dplyr::everything in combination with dplyr::select to reorder columnsdplyr::count, dplyr::tally, and dplyr::add_count and dplyr::add_tallyforcats::fct_lumpdplyr::filter_all or dplyr::filter_if in combination with dplyr::all_vars and dplyr::any_varsdplyr::group_by_if performs quick conditional groupingdplyr::rowwisepurrr trickspurrr::map_df to read in and merge all data files in a folderpurr::map_df and fs::dir_ls to read in and merge all data files following a specific pattern in a folderlist.files and purrr::map_df to read in and merge all data files in a folder broom::tidy puts your model results in a tidy data framecorrrdf %>% .$column_name or df %$% column_name can retrieve a column from a tibble dplyr::coalesce finds the one value contained in many columns with missing valuesscales::percent(myfraction)readr::parse_number(mydf$mycol) colors() to see the names of all built-in colorsGGally::ggpairs for beautiful pair-wise correlation plotstidyr::complete to get barplot spacing righttheme_set to set a default ggplot2 theme+ NULL to the end of your ggplot2 chain during developmentclip = "off" to draw outside the plot panelstroke = 0 patchwork or cowplotmagickgganimate package and "header-includes: - \usepackage{animate} in your YAML preambledata-argument in a geom_*, then it applies that function to the data!stat_function function. Normal distributions, student t-distributions, beta distributions, anything. See also here.
ggplot2 by David Robinsonggplot2 by JonTheGeekgganimate (old API) by David Robinsongganimate (updated API) by Oscar de Léongganimate (old API) by David Kretch install.packages("onion"); require(onion); data(bunny); p3d(bunny,theta=3,phi=104,box=FALSE), via R-bloggersggplot2ggtextures to make your own ggplot::geom_* such as ggbarf::geom_barfbortels::fork functionobject of type closure is not subsettable????"", via Redditexample(readLine), via DecisionStats?.Internal, via DecisionStats
Zack Nado wrote the best machine learning application I’ve seen so far: a neural network architecture that generates new Pusheen pictures.
In his blog, Zack describes his generative adversarial network (GAN) , a special type of machine learning architecture where two neural networks try to fool each other. Zack first gave the discriminator network some real Pusheen images, so it gets an idea of what Pusheen looks like. Next, the generator network gets a bunch of random numbers so it can generate completely new (fake) images. These generated images are then fed back into the discriminator, so it knows what generated images look like. Zack repeated this process several hundred thousand times, so he obtained a generator network that’s great at making new Pusheen images which the discriminator (nearly) can’t dinstinguish from the original, real ones. Below is the learning process of the generator network visualized:
In the end, the generated images are very much like the real Pusheen. Zack added an interactive module (using Tensorflow.js) to the blog so you can generate some Pusheens yourself. (it didn’t work for me though…) On a final note, Zack wrote the orginal blog both in plain English, for non-experts, and in jargon, for the more experienced data scientists. I highly recommend you read either one of those versions!
Why do groups of people act smart, dumb, kind, or cruel? People behave in strange ways, particularly when they are able to influence one another. Both good and bad things can happen when people interact and behave in network structures. On the bright side, you must be familiar with the wisdom of the crowd, where the aggregated knowledge of a group is more valuable than its sum? Ensemble algorithms – like random forest analysis – rely on this positive principle.
On the dark side, are you familiar with the phenomenon called the tragedy of the commons, where shared resource-systems collapse because individuals behave in their self-interest? Or psychological phenomena such as groupthink, where groups of people make irrational decisions due to social issues? The recent spread of fake news and misinformation is also stimulated by network interactions. In these cases, we could speak of the madness of the crowd.
Nicky Case made a great interactive walkthrough explaining why and when networks of people become wise or mad. You are tasked to change and simulate network interactions while Nicky explains concepts such as (complex) contagion, the majority illusion paradox, bonding and bridging, and small world networks. In the references, Nicky provides links to scientific papers explaining these concepts in more detail. I highly suggest you check out her website here.
paulvanderlaken.com 
