{"id":197,"date":"2022-03-21T14:00:00","date_gmt":"2022-03-21T14:00:00","guid":{"rendered":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/?p=197"},"modified":"2024-03-10T16:44:51","modified_gmt":"2024-03-10T16:44:51","slug":"the-tidyverse-the-best-verse-for-data-scientist","status":"publish","type":"post","link":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/2022\/03\/21\/the-tidyverse-the-best-verse-for-data-scientist\/","title":{"rendered":"The Tidyverse: the best* -verse for data scientist"},"content":{"rendered":"<span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 4<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>\n<p>There are a couple popular universes out there like the MCU and its multiverse, and Zuckerberg&#8217;s Metaverse. My personal favourite however is actually a universe of R packages.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.tidyverse.org\/images\/hex-tidyverse.png\" alt=\"\" width=\"253\" height=\"292\" \/><\/figure><\/div>\n\n\n\n<p>This post is by no means a tutorial for the <em>tidyverse<\/em>. Nor is it an introduction to these packages or style of coding using R. Instead, this is just a compilation of my favourite features of the packages that will hopefully convince you of its power and convert you to the tidy side.<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>What is in the <em>tidyverse<\/em>?<\/li><li>Tibbles!<\/li><li>Pipes &amp; Purrr<\/li><li>A few (more) of my favourite things<\/li><\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is in the <em>tidyverse<\/em>?<\/h2>\n\n\n\n<p>The <em>tidyverse <\/em>is a collection of R packages designed by Hadley Wickham for data science. It includes packages useful for loading, wrangling, modelling and visualising data, and a couple that make programming in R so much better. When you install and load the <em>tidyverse<\/em>, the following core packages will be loaded:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>install.packages(\"tidyverse\")\nlibrary(tidyverse)<\/code><\/pre>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<ul class=\"wp-block-list\"><li>readr &#8211; to import tabular data<\/li><li>tibble &#8211; the better* dataframe<\/li><li><span style=\"color: initial\">dplyr &#8211; for data manipulation<\/span><\/li><li>tidyr &#8211; to make data tidy<\/li><li>ggplot2 &#8211; for data visualisation<\/li><li>purrr &#8211; for functional programming<\/li><li>stringr &#8211; for string manipulation<\/li><li>forcats &#8211; for factors<\/li><\/ul>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/bookdown.org\/jkylearmstrong\/jeff_data_wrangling\/Week_1\/images\/Tidyverse_Hex_Stickers.png\" alt=\"\" width=\"310\" height=\"347\" \/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p>There are also a couple other packages that are also installed for working with specific types of vectors, importing other data types and for modelling. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Tibbles!<\/h2>\n\n\n\n<p>I must start this section by addressing the * I&#8217;ve included so far. The <em>tidyverse<\/em> developers themselves describe it as an <strong>opinionated <\/strong>collection of R packages on the <a rel=\"noreferrer noopener\" href=\"https:\/\/www.tidyverse.org\/\" target=\"_blank\">website<\/a>. So when I say that tibbles are better than data frames, that&#8217;s just my opinion as someone who has drunk the Kool-Aid and loves it.<\/p>\n\n\n\n<p>If you&#8217;ve ever used the data.frame or data.table, unless you&#8217;ve completely mastered using them, you may agree with me that it can be a bit confusing remembering how many commas are needed, whether to use square brackets or parentheses, if something is being done in place or if you need to make a copy. A <strong>tibble <\/strong>is &#8220;a modern reimagining of the data.frame&#8221;. The developers put it nicely when they said that tibbles are lazy and surly data.frames: they do less and complain more.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Pipes &amp; Purrr<\/h2>\n\n\n\n<p>When you&#8217;re trying to manipulate data, or doing analysis that isn&#8217;t super simple, it&#8217;s very likely that you&#8217;ll end up with nested functions. Here&#8217;s a simple example:  you want to create a table of random numbers using different distributions and then add a column for a new distribution.<\/p>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:33.33%\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/magrittr.tidyverse.org\/logo.png\" alt=\"\" \/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>There are a couple ways to approach this: you could create a new variable (or overwrite the variable) at each step. Or you could use pipes. Among other things, piping saves you having to rewrite variable names, avoid nested function calls and makes code look so much more <em>elegant<\/em>. The pipe operator <strong>%&gt;% <\/strong>is included when you install the <em>tidyverse<\/em>. <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#both bits of code do the same thing\n\nno_pipe = tibble(N = rnorm(10), E = rexp(10))\nno_pipe = mutate(no_pipe, G = rgamma(10,1))\n\nwith_pipe = \n  tibble(N = rnorm(10), E = rexp(10)) %&gt;%\n  mutate(G = rgamma(10,1))<\/code><\/pre>\n<\/div>\n<\/div>\n\n\n\n<p>If you want to take it to the next level, the package <em>magrittr<\/em> includes several other piping operators. My personal favourite is the assignment pipe <strong>%&lt;&gt;%<\/strong> which allows you to modify data in place. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. A few more of my favourite things<\/h2>\n\n\n\n<p>As I said at the start, this is by not meant to be a comprehensive introduction to the <em>tidyverse<\/em>. Now that I&#8217;ve introduced a few of the basics, here are a couple other features (each of which could have its own post really) that make these packages so great:<\/p>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading\">Tibbles and what they can store<\/h5>\n\n\n\n<p>With tibbles, the columns type does not have to be a core data type. As well as the basics (integer, numeric, string, factor, logical), cells can contain vectors, lists, tibbles or almost anything really. And you can move between complex and simple data types using functions in the package <em>dplyr<\/em> has useful functions to nest, unnest and pivot data to the desired shape without much hassle.<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading\">Purrr, map and all its variants<\/h5>\n\n\n\n<p>You can make code a lot easier to read by using map functions to replace for loops. This is really helpful when you have nested tibbles that you want to perform a set of operations over.<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading\">Tidyselect and dplyr<\/h5>\n\n\n\n<p>The group_by function from <em>dplyr <\/em>and the many helper functions included in the package <em>tidyselect<\/em> make summarising and manipulating groups of data super straightforward. <\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading\">Grammar of Graphics<\/h5>\n\n\n\n<p>I&#8217;ll admit that when you just start using ggplot2 in R, it may seem really complicated, especially when compared to the base graphics package included in R. But once you get a hang of the basics, you can create some spectacular visualisations.<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<h5 class=\"wp-block-heading\">R Markdown<\/h5>\n\n\n\n<p>R Markdown is an amazing way to combine code, results and commentary and save them as accessible file types. It is really good as both a lab notebook to keep track of your work and thoughts, and as a means of communicating every step of the analysis process. <\/p>\n<\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">Learn More<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>This <a rel=\"noreferrer noopener\" href=\"https:\/\/r4ds.had.co.nz\/\" data-type=\"URL\" data-id=\"https:\/\/r4ds.had.co.nz\/\" target=\"_blank\">online book<\/a> (as well as lots of practice) is where I learnt most of what I know about the <em>tidyverse<\/em>. <\/li><li>A really helpful <a href=\"https:\/\/github.com\/rstudio\/cheatsheets\/blob\/main\/data-visualization-2.1.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">cheatsheet<\/a> for ggplot2<\/li><\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"span-reading-time rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">Reading Time: <\/span> <span class=\"rt-time\"> 4<\/span> <span class=\"rt-label rt-postfix\">minutes<\/span><\/span>This post is by no means a tutorial for the tidyverse. Nor is it an introduction to these packages or style of coding using R. Instead, this is just a compilation of my favourite features of the packages that will hopefully convince you of its power and convert you to the tidy side.<\/p>\n","protected":false},"author":41,"featured_media":285,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[13],"tags":[],"class_list":{"0":"post-197","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","6":"hentry","7":"category-a-few-of-my-favourite-things","9":"post-with-thumbnail","10":"post-with-thumbnail-large"},"_links":{"self":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/posts\/197","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/comments?post=197"}],"version-history":[{"count":14,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/posts\/197\/revisions"}],"predecessor-version":[{"id":283,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/posts\/197\/revisions\/283"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/media\/285"}],"wp:attachment":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/media?parent=197"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/categories?post=197"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/danielle-notice\/wp-json\/wp\/v2\/tags?post=197"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}