Graphing With R

I took a few days off for Christmas in San Antonio with my parents and my sister. While there, I had the usual tasks for a geek with non-geek relatives: fix computers, fix wifi, fix printers, explain the Internet, explain how stuff works. But in between time spent doing that, and doing family stuff, I learned a bit about R. R is programming language mostly intended to do statistical programming. I don’t know much about that, but I wanted a way to make some graphs that are sometimes difficult to make in Excel.

I think it would be easier to just learn more about Excel, but now I am invested so I am trying to use it as much as possible. Recently at work I added a post to our blog on the impact of JavaScript and concatenating/minifying the code on your website. You can read the about it here At the end of that post, I added an chart describing the final results.

JavaScript tests chart built with R

JavaScript tests chart built with R

Creating a chart like this would probably take me 20-30 minutes in Excel. Building it in R would probably take less time if I knew R like I know Excel. Instead it took me a lot longer. But in getting there I discovered a lot of ways I didn’t like the image. I think Excel is great when you know exactly how you want the final image to appear in the post. I thought I did, but this looks vastly different from that initial version. Tweaking and then having a very repeatable chart within seconds for any dataset is one of R‘s biggest strengths for me.

So how did I get here? Well, R is complicated. Luckily a course on Coursera came up at exactly the right time for me. There is a second course that takes this further that starts next week too.

But assuming that you know the basics of R and just want to see what I did, here is my R Script for this chart (the csv files I am using are the standard ones you can download after running a test on Yottaa):

body, td {
font-family: sans-serif;
background-color: white;
font-size: 12px;
margin: 8px;
}

tt, code, pre {
font-family: ‘DejaVu Sans Mono’, ‘Droid Sans Mono’, ‘Lucida Console’, Consolas, Monaco, monospace;
}

h1 {
font-size:2.2em;
}

h2 {
font-size:1.8em;
}

h3 {
font-size:1.4em;
}

h4 {
font-size:1.0em;
}

h5 {
font-size:0.9em;
}

h6 {
font-size:0.8em;
}

a:visited {
color: rgb(50%, 0%, 50%);
}

pre {
margin-top: 0;
max-width: 95%;
border: 1px solid #ccc;
white-space: pre-wrap;
}

pre code {
display: block; padding: 0.5em;
}

code.r, code.cpp {
background-color: #F8F8F8;
}

table, td, th {
border: none;
}

blockquote {
color:#666666;
margin:0;
padding-left: 1em;
border-left: 0.5em #EEE solid;
}

hr {
height: 0px;
border-bottom: none;
border-top-width: thin;
border-top-style: dotted;
border-top-color: #999999;
}

@media print {
* {
background: transparent !important;
color: black !important;
filter:none !important;
-ms-filter: none !important;
}

body {
font-size:12pt;
max-width:100%;
}

a, a:visited {
text-decoration: underline;
}

hr {
visibility: hidden;
page-break-before: always;
}

pre, blockquote {
padding-right: 1em;
page-break-inside: avoid;
}

tr, img {
page-break-inside: avoid;
}

img {
max-width: 100% !important;
}

@page :left {
margin: 15mm 20mm 15mm 10mm;
}

@page :right {
margin: 15mm 10mm 15mm 20mm;
}

p, h2, h3 {
orphans: 3; widows: 3;
}

h2, h3 {
page-break-after: avoid;
}
}

pre .operator,
pre .paren {
color: rgb(104, 118, 135)
}

pre .literal {
color: rgb(88, 72, 246)
}

pre .number {
color: rgb(0, 0, 205);
}

pre .comment {
color: rgb(76, 136, 107);
}

pre .keyword {
color: rgb(0, 0, 255);
}

pre .identifier {
color: rgb(0, 0, 0);
}

pre .string {
color: rgb(3, 106, 7);
}

var hljs=new function(){function m(p){return p.replace(/&/gm,”&”).replace(/</gm,"<")}function f(r,q,p){return RegExp(q,"m"+(r.cI?"i":"")+(p?"g":""))}function b(r){for(var p=0;p<r.childNodes.length;p++){var q=r.childNodes[p];if(q.nodeName=="CODE"){return q}if(!(q.nodeType==3&&q.nodeValue.match(/s+/))){break}}}function h(t,s){var p="";for(var r=0;r<t.childNodes.length;r++){if(t.childNodes[r].nodeType==3){var q=t.childNodes[r].nodeValue;if(s){q=q.replace(/n/g,"")}p+=q}else{if(t.childNodes[r].nodeName=="BR"){p+="n"}else{p+=h(t.childNodes[r])}}}if(/MSIE [678]/.test(navigator.userAgent)){p=p.replace(/r/g,"n")}return p}function a(s){var r=s.className.split(/s+/);r=r.concat(s.parentNode.className.split(/s+/));for(var q=0;q<r.length;q++){var p=r[q].replace(/^language-/,"");if(e[p]){return p}}}function c(q){var p=[];(function(s,t){for(var r=0;r<s.childNodes.length;r++){if(s.childNodes[r].nodeType==3){t+=s.childNodes[r].nodeValue.length}else{if(s.childNodes[r].nodeName=="BR"){t+=1}else{if(s.childNodes[r].nodeType==1){p.push({event:"start",offset:t,node:s.childNodes[r]});t=arguments.callee(s.childNodes[r],t);p.push({event:"stop",offset:t,node:s.childNodes[r]})}}}}return t})(q,0);return p}function k(y,w,x){var q=0;var z="";var s=[];function u(){if(y.length&&w.length){if(y[0].offset!=w[0].offset){return(y[0].offset<w[0].offset)?y:w}else{return w[0].event=="start"?y:w}}else{return y.length?y:w}}function t(D){var A="<"+D.nodeName.toLowerCase();for(var B=0;B”}while(y.length||w.length){var v=u().splice(0,1)[0];z+=m(x.substr(q,v.offset-q));q=v.offset;if(v.event==”start”){z+=t(v.node);s.push(v.node)}else{if(v.event==”stop”){var p,r=s.length;do{r–;p=s[r];z+=(“”)}while(p!=v.node);s.splice(r,1);while(r<s.length){z+=t(s[r]);r++}}}}return z+m(x.substr(q))}function j(){function q(x,y,v){if(x.compiled){return}var u;var s=[];if(x.k){x.lR=f(y,x.l||hljs.IR,true);for(var w in x.k){if(!x.k.hasOwnProperty(w)){continue}if(x.k[w] instanceof Object){u=x.k[w]}else{u=x.k;w="keyword"}for(var r in u){if(!u.hasOwnProperty(r)){continue}x.k[r]=[w,u[r]];s.push(r)}}}if(!v){if(x.bWK){x.b="\b("+s.join("|")+")\s"}x.bR=f(y,x.b?x.b:"\B|\b");if(!x.e&&!x.eW){x.e="\B|\b"}if(x.e){x.eR=f(y,x.e)}}if(x.i){x.iR=f(y,x.i)}if(x.r===undefined){x.r=1}if(!x.c){x.c=[]}x.compiled=true;for(var t=0;t<x.c.length;t++){if(x.c[t]=="self"){x.c[t]=x}q(x.c[t],y,false)}if(x.starts){q(x.starts,y,false)}}for(var p in e){if(!e.hasOwnProperty(p)){continue}q(e[p].dM,e[p],true)}}function d(B,C){if(!j.called){j();j.called=true}function q(r,M){for(var L=0;L<M.c.length;L++){if((M.c[L].bR.exec(r)||[null])[0]==r){return M.c[L]}}}function v(L,r){if(D[L].e&&D[L].eR.test(r)){return 1}if(D[L].eW){var M=v(L-1,r);return M?M+1:0}return 0}function w(r,L){return L.i&&L.iR.test(r)}function K(N,O){var M=[];for(var L=0;L<N.c.length;L++){M.push(N.c[L].b)}var r=D.length-1;do{if(D[r].e){M.push(D[r].e)}r–}while(D[r+1].eW);if(N.i){M.push(N.i)}return f(O,M.join("|"),true)}function p(M,L){var N=D[D.length-1];if(!N.t){N.t=K(N,E)}N.t.lastIndex=L;var r=N.t.exec(M);return r?[M.substr(L,r.index-L),r[0],false]:[M.substr(L),"",true]}function z(N,r){var L=E.cI?r[0].toLowerCase():r[0];var M=N.k[L];if(M&&M instanceof Array){return M}return false}function F(L,P){L=m(L);if(!P.k){return L}var r="";var O=0;P.lR.lastIndex=0;var M=P.lR.exec(L);while(M){r+=L.substr(O,M.index-O);var N=z(P,M);if(N){x+=N[1];r+='‘+M[0]+”“}else{r+=M[0]}O=P.lR.lastIndex;M=P.lR.exec(L)}return r+L.substr(O,L.length-O)}function J(L,M){if(M.sL&&e[M.sL]){var r=d(M.sL,L);x+=r.keyword_count;return r.value}else{return F(L,M)}}function I(M,r){var L=M.cN?’‘:””;if(M.rB){y+=L;M.buffer=””}else{if(M.eB){y+=m(r)+L;M.buffer=””}else{y+=L;M.buffer=r}}D.push(M);A+=M.r}function G(N,M,Q){var R=D[D.length-1];if(Q){y+=J(R.buffer+N,R);return false}var P=q(M,R);if(P){y+=J(R.buffer+N,R);I(P,M);return P.rB}var L=v(D.length-1,M);if(L){var O=R.cN?”“:””;if(R.rE){y+=J(R.buffer+N,R)+O}else{if(R.eE){y+=J(R.buffer+N,R)+O+m(M)}else{y+=J(R.buffer+N+M,R)+O}}while(L>1){O=D[D.length-2].cN?”“:””;y+=O;L–;D.length–}var r=D[D.length-1];D.length–;D[D.length-1].buffer=””;if(r.starts){I(r.starts,””)}return R.rE}if(w(M,R)){throw”Illegal”}}var E=e[B];var D=[E.dM];var A=0;var x=0;var y=””;try{var s,u=0;E.dM.buffer=””;do{s=p(C,u);var t=G(s[0],s[1],s[2]);u+=s[0].length;if(!t){u+=s[1].length}}while(!s[2]);if(D.length>1){throw”Illegal”}return{r:A,keyword_count:x,value:y}}catch(H){if(H==”Illegal”){return{r:0,keyword_count:0,value:m(C)}}else{throw H}}}function g(t){var p={keyword_count:0,r:0,value:m(t)};var r=p;for(var q in e){if(!e.hasOwnProperty(q)){continue}var s=d(q,t);s.language=q;if(s.keyword_count+s.r>r.keyword_count+r.r){r=s}if(s.keyword_count+s.r>p.keyword_count+p.r){r=p;p=s}}if(r.language){p.second_best=r}return p}function i(r,q,p){if(q){r=r.replace(/^((]+>|t)+)/gm,function(t,w,v,u){return w.replace(/t/g,q)})}if(p){r=r.replace(/n/g,”
“)}return r}function n(t,w,r){var x=h(t,r);var v=a(t);var y,s;if(v){y=d(v,x)}else{return}var q=c(t);if(q.length){s=document.createElement(“pre”);s.innerHTML=y.value;y.value=k(q,c(s),x)}y.value=i(y.value,w,r);var u=t.className;if(!u.match(“(\s|^)(language-)?”+v+”(\s|$)”)){u=u?(u+” “+v):v}if(/MSIE [678]/.test(navigator.userAgent)&&t.tagName==”CODE”&&t.parentNode.tagName==”PRE”){s=t.parentNode;var p=document.createElement(“div”);p.innerHTML=”

”;t=p.firstChild.firstChild;p.firstChild.cN=s.cN;s.parentNode.replaceChild(p.firstChild,s)}else{t.innerHTML=y.value}t.className=u;t.result={language:v,kw:y.keyword_count,re:y.r};if(y.second_best){t.second_best={language:y.second_best.language,kw:y.second_best.keyword_count,re:y.second_best.r}}}function o(){if(o.called){return}o.called=true;var r=document.getElementsByTagName(“pre”);for(var p=0;p<r.length;p++){var q=b(r[p]);if(q){n(q,hljs.tabReplace)}}}function l(){if(window.addEventListener){window.addEventListener("DOMContentLoaded",o,false);window.addEventListener("load",o,false)}else{if(window.attachEvent){window.attachEvent("onload",o)}else{window.onload=o}}}var e={};this.LANGUAGES=e;this.highlight=d;this.highlightAuto=g;this.fixMarkup=i;this.highlightBlock=n;this.initHighlighting=o;this.initHighlightingOnLoad=l;this.IR="[a-zA-Z][a-zA-Z0-9_]*";this.UIR="[a-zA-Z_][a-zA-Z0-9_]*";this.NR="\b\d+(\.\d+)?";this.CNR="\b(0[xX][a-fA-F0-9]+|(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?)";this.BNR="\b(0b[01]+)";this.RSR="!|!=|!==|%|%=|&|&&|&=|\*|\*=|\+|\+=|,|\.|-|-=|/|/=|:|;|<|<<|<<=||>=|>>|>>=|>>>|>>>=|\?|\[|\{|\(|\^|\^=|\||\|=|\|\||~”;this.ER=”(?![\s\S])”;this.BE={b:”\\.”,r:0};this.ASM={cN:”string”,b:”‘”,e:”‘”,i:”\n”,c:[this.BE],r:0};this.QSM={cN:”string”,b:'”‘,e:'”‘,i:”\n”,c:[this.BE],r:0};this.CLCM={cN:”comment”,b:”//”,e:”$”};this.CBLCLM={cN:”comment”,b:”/\*”,e:”\*/”};this.HCM={cN:”comment”,b:”#”,e:”$”};this.NM={cN:”number”,b:this.NR,r:0};this.CNM={cN:”number”,b:this.CNR,r:0};this.BNM={cN:”number”,b:this.BNR,r:0};this.inherit=function(r,s){var p={};for(var q in r){p[q]=r[q]}if(s){for(var q in s){p[q]=s[q]}}return p}}();hljs.LANGUAGES.cpp=function(){var a={keyword:{“false”:1,”int”:1,”float”:1,”while”:1,”private”:1,”char”:1,”catch”:1,”export”:1,virtual:1,operator:2,sizeof:2,dynamic_cast:2,typedef:2,const_cast:2,”const”:1,struct:1,”for”:1,static_cast:2,union:1,namespace:1,unsigned:1,”long”:1,”throw”:1,”volatile”:2,”static”:1,”protected”:1,bool:1,template:1,mutable:1,”if”:1,”public”:1,friend:2,”do”:1,”return”:1,”goto”:1,auto:1,”void”:2,”enum”:1,”else”:1,”break”:1,”new”:1,extern:1,using:1,”true”:1,”class”:1,asm:1,”case”:1,typeid:1,”short”:1,reinterpret_cast:2,”default”:1,”double”:1,register:1,explicit:1,signed:1,typename:1,”try”:1,”this”:1,”switch”:1,”continue”:1,wchar_t:1,inline:1,”delete”:1,alignof:1,char16_t:1,char32_t:1,constexpr:1,decltype:1,noexcept:1,nullptr:1,static_assert:1,thread_local:1,restrict:1,_Bool:1,complex:1},built_in:{std:1,string:1,cin:1,cout:1,cerr:1,clog:1,stringstream:1,istringstream:1,ostringstream:1,auto_ptr:1,deque:1,list:1,queue:1,stack:1,vector:1,map:1,set:1,bitset:1,multiset:1,multimap:1,unordered_set:1,unordered_map:1,unordered_multiset:1,unordered_multimap:1,array:1,shared_ptr:1}};return{dM:{k:a,i:”</",c:[hljs.CLCM,hljs.CBLCLM,hljs.QSM,{cN:"string",b:"'\\?.",e:"'",i:"."},{cN:"number",b:"\b(\d+(\.\d*)?|\.\d+)(u|U|l|L|ul|UL|f|F)"},hljs.CNM,{cN:"preprocessor",b:"#",e:"$"},{cN:"stl_container",b:"\b(deque|list|queue|stack|vector|map|set|bitset|multiset|multimap|unordered_map|unordered_set|unordered_multiset|unordered_multimap|array)\s*”,k:a,r:10,c:[“self”]}]}}}();hljs.LANGUAGES.r={dM:{c:[hljs.HCM,{cN:”number”,b:”\b0[xX][0-9a-fA-F]+[Li]?\b”,e:hljs.IMMEDIATE_RE,r:0},{cN:”number”,b:”\b\d+(?:[eE][+\-]?\d*)?L\b”,e:hljs.IMMEDIATE_RE,r:0},{cN:”number”,b:”\b\d+\.(?!\d)(?:i\b)?”,e:hljs.IMMEDIATE_RE,r:1},{cN:”number”,b:”\b\d+(?:\.\d*)?(?:[eE][+\-]?\d*)?i?\b”,e:hljs.IMMEDIATE_RE,r:0},{cN:”number”,b:”\.\d+(?:[eE][+\-]?\d*)?i?\b”,e:hljs.IMMEDIATE_RE,r:1},{cN:”keyword”,b:”(?:tryCatch|library|setGeneric|setGroupGeneric)\b”,e:hljs.IMMEDIATE_RE,r:10},{cN:”keyword”,b:”\.\.\.”,e:hljs.IMMEDIATE_RE,r:10},{cN:”keyword”,b:”\.\.\d+(?![\w.])”,e:hljs.IMMEDIATE_RE,r:10},{cN:”keyword”,b:”\b(?:function)”,e:hljs.IMMEDIATE_RE,r:2},{cN:”keyword”,b:”(?:if|in|break|next|repeat|else|for|return|switch|while|try|stop|warning|require|attach|detach|source|setMethod|setClass)\b”,e:hljs.IMMEDIATE_RE,r:1},{cN:”literal”,b:”(?:NA|NA_integer_|NA_real_|NA_character_|NA_complex_)\b”,e:hljs.IMMEDIATE_RE,r:10},{cN:”literal”,b:”(?:NULL|TRUE|FALSE|T|F|Inf|NaN)\b”,e:hljs.IMMEDIATE_RE,r:1},{cN:”identifier”,b:”[a-zA-Z.][a-zA-Z0-9._]*\b”,e:hljs.IMMEDIATE_RE,r:0},{cN:”operator”,b:”|=||<|!|&|\||\$|:",e:hljs.IMMEDIATE_RE,r:0},{cN:"operator",b:"%",e:"%",i:"\n",r:1},{cN:"identifier",b:"",e:"",r:0},{cN:"string",b:'"',e:'"',c:[hljs.BE],r:0},{cN:"string",b:"'",e:"'",c:[hljs.BE],r:0},{cN:"paren",b:"[[({\])}]",e:hljs.IMMEDIATE_RE,r:0}]}};
hljs.initHighlightingOnLoad();

I am having a lot of fun with R so far and I look forward to seeing what I come up with over the next few weeks. I have a series of posts coming up on the Yottaa Blog and plenty of opportunity to post more charts made with R.

The Mac and The Watch

I used to wear my watch all the time. It’s a Seiko Sportura I picked up on one of my first trips to Dubai. It was the first watch I paid over 50 bucks for and at closer to 400-500 dollars, it was a fairly big investment. It stayed on my wrist all the time. I wore it everywhere for at least 6 years and when it was off, the lack of weight on my wrist just felt wierd. 

But if you look at my arm today, you might notice that the watch isn’t there. I don’t wear it anymore. It’s not that I don’t like the watch anymore, leaving it in a drawer for the last year. What changed it for me was the Mac. So at this point you are probably wondering what the Mac offers that caused me to stop wearing the watch. And the answer is nothing. The reason is because Mac made it painful.

The Sportura is a big watch. I have the black titanium wrist band which makes it feel even bigger. Everything was fine until I went to the Macbook Pro and its poorly designed wrist rest area. That hard, razor-sharp 90 degree angle at the front of the laptop would always catch the bottom of the watch wrist band. I would have to lift my arm and then rest it on the laptop. Every movement would then make a scratching sound on the aluminum. 

Compare that with the more beveled front of the Thinkpad I had used before: my wrist could slide on to the laptop without it catching. Rubbing the watch on the Thinkpad caused no scratching sound. And all of that makes sense based on what I have seen about how design is done by Apple vs the Thinkpad group at Lenovo. Apple makes something beautiful regardless of it’s users or it’s history. While the Thinkpads are an evolution of design with each generation building on the greatness of the earlier models. I can still see some aspects of that original butterfly keyboard in the latest Thinkpads and its incredible. 

I am 100% an Apple guy today but that doesn’t mean I think everything they do is perfect. I spend all of my time on a Macbook Pro but that doesn’t mean I can’t see the genius in the Thinkpad. In my perfect world, the Apple designers would find a way to integrate a more beveled front edge into the Macbook (and a thousand bonus points for implementing the Thinkpad keyboard with the Trackstick). Until then, the watch will stay off, sitting in a drawer somewhere.

Learning R With Coursera

Last week I started taking a new course on Coursera.org. It’s called Computing for Data Analysis and its all about using the R programming language to get a better understanding of large datasets. I work for Yottaa and large datasets are something we deal a lot with. I want to find an interesting way to work with and understand the data collected from all sorts of sites on the internet and I am hoping R is the perfect solution. Unfortunately, its a bit wierd and not all that well documented, so this training course could be just what I need.

The programming assignment from last week already taught me a few commands I hadn’t used before. We are working with a table of numbers describing ozone, temperature and more and we needed to find the mean of the Solar.R value where Ozone was greater than 31 and temperature was greater than 90. At first it took me about 7 lines of code to get this done, but after some searching, managed to get it down to a single beautiful line (which I am sure I would forget if I did not document it here):

Wow, that is cool!

I am looking forward to the second section of the course which starts on Wednesday!