October 28, 2017

Deciding I Need To Use AWS CodeBuild

If you at least partially live in the Amazon Web Services (AWS) world, then you know that re:INVENT is coming up in just a few weeks. That's the big show for the industry with 50,000+ expected attendees this year. The company I work for has an enormous presence, and for the 3rd year in a row, I have a speaking slot. Last year I had 1800 folks in the room listening to me introduce AWS Lambda to them. This time I am taking the next step to talk about orchestration with AWS Step Functions.

Last year my demo was pretty cool. But this year I want to do something even better. I am still building it out but wanted to start giving you some hints about (and self document) the components I am using. At the heart of it all is a GatsbyJS-based statically generated website. Setting up the site is easy enough. Even compiling it is super easy. But getting that compile to happen in the cloud is a little more involved.

The primary reason for setting this up is that I need a cool demo. But recently I was in Italy for vacation. My wife was so proud of me for leaving the laptop home, but that meant I could not publish any of the blog posts I wrote till I returned to the laptop that had the Hugo source code on it. And then I forgot I wrote them until 20 minutes ago. Having a service in the cloud to do this for me would have solved the problem. Now there are many ways to solve this, but this is for a re:INVENT demo and I live in that world.

OK, so what is CodeBuild? Well CodeBuild is a serverless offering from AWS for running code builds. Essentially serverless just refers to any service where you don't need to worry about the infrastructure, but you have the power and flexibility to do almost anything you like. And perhaps most importantly, you don't pay for it when it's not doing anything for you. If you run it for an entire month, an EC2 is going to be cheaper, but this is for workloads you use for seconds every hour or day.

CodeBuild is priced per minute of use and the cost starts at half a penny per minute. Thats over $200 a month if its running 24/7, but you are only using this for a few minutes a day. And the first 100 minutes per month are free.

What can you do with a CodeBuild instance? Anything you like. There are preconfigured environments for Docker, NodeJS, PHP, Ruby, Android, Go, Java, Python and more. You point it at your code repository on GitHub, BitBucket, AWS CodeCommit. Heck, I think you could even do a KeyBase git repo. You aren't going to want to run a PHP website from here because it takes a few seconds to spin up. But if you are doing a compile, it's perfect.

My use case is a NodeJS static website generator. My source code is hosted on AWS CodeCommit. When I start a compile, it spins up the environment, clone's the repo, installs any needed NPM modules, and runs the build command. When it's done it pushes the results to an S3 bucket. And for me, S3 is the storage for my demo website, via AWS CloudFront.

There are a bunch of environments you can choose from, but to make things cooler, you can provide your own environment in the form of a Docker image. And that image can be hosted on Docker Hub, AWS ECS, or any other image repository (it's re:INVENT, can you guess where my image is).

The big benefit for me to use my own Docker image is that the prerequisite NPM modules are already installed so I don't have to wait an extra minute or two for that step. So I was able to drop a 3+ minute build down to less than one minute.

I'll show you how I built all this out in the next post and in future posts I'll automate the build using a Lambda listening to the CodeCommit repo, and add automated testing with other Lambdas via Step Functions.

If you want to see the full demo done in my super stylish way, then register for my session at re:INVENT. It's full now, but that happened last year too and they kept bumping me up to bigger rooms. Hopefully it will happen again this year, so sign up for the waitlist. My talk is SRV335 - Best Practices for Orchestrating AWS Lambda Workloads

July 31, 2017

Moved Technovangelist to a New Site

This weekend I built a new site. Why? Not sure. I have lots of reasons, but none are so strong that they are the main reason to do it. I moved from Wordpress on Digital Ocean to Hugo on

Why get off Wordpress?

Wordpress is a great service if you have the time to maintain (or have someone doing it for you). And its great to host a lot of different sites on a single server. And its a great dynamic site engine. I hear a lot of folks say it limited and has a lot of issues, but they tend to be folks that don't invest the time in doing Wordpress right.

Wordpress has one of the same big weaknesses of Windows: it's too easy to use. Folks see that ease and think its easy all the way through. Below the surface, it's hard...and it's surprisingly easy to really screw things up. One misplaced semicolon, even in a plugin, and the site is down.

The reason I went to Wordpress is that after leaving Placester I decided to do my own thing. I had been helping Placester partners build on the Placester Wordpress plugin, so decided it would be easy to start building Wordpress themes. And so I needed to host a Wordpress site to show that I knew what I was doing.

But Wordpress, if you are doing it all on your own, means hosting a server and maintaining it. I am now in a phase where I don't really want to maintain a little server to host Wordpress. And every few days the server was falling down.

Why get off Digital Ocean?

Digital Ocean is also a great service. I used them a lot at Placester (I think the founders might have been in the same Techstars class or something like that). Then there was the deal where if you subscribed to the Changelog podcast, you got 50 bucks credit at DO. So that paid for a lot of months at $5 per month.

Droplets on DO are amazing. I set one up 6 years ago and its still going strong. But even $5 is a bit to pay if you aren't really doing anything with it. And the thing I was doing kept falling down.

Why use Hugo?

My site is nothing special. I do nothing dynamic. I sell nothing. I have no need for Wordpress or Drupal or anything else with a database behind the scenes. A static site would serve all my needs quite well. There are plenty of choices for static site generators. Datadog's marketing site and now our docs are built with Hugo, so it makes a lot of sense to use.

But I am investing a bit of time getting better at Javascript and Typescript. I want to focus that into a static site as well. So I first picked Gatsby. I spent almost a full weekend getting it up and running. But after adding my 300 or so posts, it got so slow and often ran out of memory. My site isn't all that big, but it will slowly grow. And if it's having trouble where I start, it's only going to get worse. In 30 minutes with Hugo though, I had a fully up and running site.

Why use

A colleague uses surge for his site and hadn't nothing but good things to say about it. It's a cli only service and it fits in with the Hugo workflow. I can run hugo;surge public and a few seconds later the site is up and out there. It's pretty amazing, and for the level of service I need, it's free.

We shall see how well it performs, but I am pretty happy with it so far. And that is where things are.

December 18, 2016

Reading The Next 93 – A goal for 2017

Looking back on 2016, I think there are many accomplishments to be proud of. Speaking in front of 1800 at re:Invent is a specific highlight, but there is something else that is even more magnificent in my eyes. This past year saw my awakening to reading.

OK, I could read before. But I read for information, to learn something specific. This year I read for fun. Reading became something I enjoyed. And even better, this year was the year I discovered the power of the Public Library. Yes, I had been to libraries before, but I did not love them like I do now. Today my local library is the Connolly branch of the Boston Public Library system and in a few months I will be able to start using the library almost across the street in Jamaica Plain (it’s been under renovation since we moved in to the neighborhood). I am so excited.

This year I think I read about 50-60 books. Before, the average for the year might be 1 or 2. But I couldn’t tell you what most of this year’s books were. Sure, “Fire in the Valley” and “Deep Work” stick out in my mind. So does the amazing story of a possible CIA agent captured in the Iranian revolution called “Off the Radar”, written by his son. Yes, those stick out, but the rest are mostly gone. Boston Public Library has a feature that will record your checkout history, but it doesn’t work. GoodReads has an integration with the BPL app, but its a manual step that I forget to click. I need to get better at this.

So that is a goal for 2017. The goal is to write a little summary of what I read. This will force the memory to last a bit longer than it does today. And the corollary goal is to read the books that I have. Sure, the library will continue to tempt me with stories and facts but I have my own library that I have neglected for years. So I have created a list of books that I want to tackle for 2017. Some are books that are currently checked out from the library, others are books we own. Some came from me, others from my wife. Some I have read a few chapters of, others have sat unopened for decades. I intend to read them. All of them?? I don’t know. But I would like to review this list in a year and see how I did. So here is the list for the me of 12 months forth:

Books that my wife and I have checked out right now that I am eager to read

  • The Making of the Atomic Bomb – Richard Rhodes
  • Titan: The Life of John D. Rockefeller, Sr. – Ron Chernow
  • Creative Schools: The Grassroots Revolution That’s Transforming Education – Ken Robinson
  • Hero of the Empire: The Boer War, a Daring Escape, and the Making of Winston Churchill – Candice Millard
  • Song of the Dodo: Island Biogeography in an Age of Extinctions — David Quammen
  • The Fish That Ate the Whale: The Life and Times of America’s Banana King – Rich Cohen
  • How Not to Be Wrong: The Power of Mathematical Thinking – Jordan Ellenberg
  • A Mind For Numbers: How to Excel at Math and Science (Even If You Flunked Algebra) – Barbara Oakley
  • The Lost Art of Finding Our Way – John Edward Huth
  • The Pigeon Tunnel: Stories from My Life – John Le Carre
  • Pirate – Clive Cussler
  • Superforcasting: The Art and Science of Prediction
  • Craft of Research – Wayne Booth
  • HBR on Innovation – HBR
  • The Last Kingdom – Bernard Cornwell
  • Books we own and I want to reread

  • Accidental Genius: Using Writing to Generate Your Best Ideas, Insight, and Content – Mark Levy
  • The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography – Simon Singh
  • Deep Work: Rules for Focused Success in a Distracted World – Cal Newport
  • Smarter Faster Better: The Secrets of Being Productive in Life and Business – Charles Duhigg
  • # Books we own, I have tried to read, but never finished
  • Black Tulip – Alexandre Dumas
  • The Anatomy of Error: Ancient Military Disasters and Their Lessons for Modern Strategists – Barry Strauss
  • Arabian Sands: Revised Edition – Wilfred Thesiger
  • Long Way Down – Ewan McGregor and Charley Boorman
  • Enders Game – Orson Scott Card
  • Hyperion – Dan Simmons
  • Books that we own but I never tried

  • Being Mortal: Medicine and What Matters in the End – Atul Gawande
  • The Book of Strange New Things: A Novel – Michel Faber
  • Golem and the Jinni – Helene Wecker
  • Gene: An Intimate History – Siddhartha Mukherjee
  • Emergence: The Connected Lives of Ants, Brains, Cities, and Software – Steven Johnson
  • Eden on the Charles: The Making of Boston – Michael Rawson
  • Moor’s Account – Laila Lalami
  • Power of Habit: Why We Do What We Do in Life and Business — Charles Duhigg
  • The Paying Guests – Sarah Waters
  • Papercuts Anthology: What Happened Here, Volume 1 – (our local bookstore)
  • Hidden Life of Trees: What They Feel, How They Communicate—Discoveries from a Secret World – Peter Wohlleben
  • Race Underground: Boston, New York, and the Incredible Rivalry That Built America’s First Subway – Doug Most
  • Power of One: A Novel – Bryce Courtenay
  • The Winter People – Jennifer McMahon
  • The Writers Portable Mentor – Priscilla Long
  • Wanderlust: A History of Walking – Rebecca Solnit
  • Innocent Abroad: Life-Changing Trips from 35 Great Writers (Anthology) – John Berendt, Dave Eggers
  • Oregon Trail: A New American Journey – Rinker Buck
  • Cryptonomicon – Neal Stephenson
  • The CS Detective – Jeremy Kubica
  • Books that came from me before marriage but I never read

  • The Land That Time Forgot – Edgar Rice Burroughs
  • A Year in Provence – Peter Mayle
  • A Tramp Abroad – Mark Twain
  • Blue Latitudes: Boldly Going Where Captain Cook Has Gone Before – Tony Horwitz
  • The Back of the Napkin: Solving Problems and Selling Ideas with Pictures – Dan Roam
  • The Selfish Gene – Richard Dawkins
  • A Study of History, Vol. 1: Abridgement of Volumes I-VI – Arnold J. Toynbee
  • Snow Falling on Cedars: A Novel – David Guterson
  • Stuff: The Things The World Is Made Of – Ivan Amato
  • The Friar and the Cipher: Roger Bacon and the Unsolved Mystery of the Most Unusual Manuscript in the World – Lawrence Goldstone, Nancy Goldstone
  • Ever Since Darwin – Stephen Jay Gould
  • Lead the Field – Earl Nightingale
  • A Mathematician Reads the Newspaper – John Allen Paulos
  • Mediterranean Winter – Robert D Kaplan
  • Our Man in Havana: An Entertainment – Graham Greene
  • The Hotel on the Roof of the World: Five Years in Tibet – Alec Le Sueur
  • The Koran
  • Herodotus: The Histories
  • How the Irish Saved Civilization – Thomas Cahill
  • The Complete Sherlock Holmes, Volume I – Arthur Conan Doyle
  • Connections – James Burke
  • The Hedgehog, the Fox, and the Magister’s Pox: Mending the Gap Between Science and the Humanities – Stephen Jay Gould
  • Guns, Germs, and Steel : The Fates of Human Societies – Jared M. Diamond
  • Great Railway Bazaar: By Train Through Asia – Paul Theroux
  • Rendezvous with Rama – Isaac Asimov
  • The Razor’s Edge – W. Somerset Maugham
  • The Prince – Niccolo Machiavelli
  • The River War – Winston S. Churchill
  • Creative Habit: Learn It and Use It for Life – Twyla Tharp
  • God: A Biography – Jack Miles
  • Books that came from Toni before marriage but I haven’t had a chance to read

  • Thinkertoys: A Handbook of Creative-Thinking Techniques (2nd Edition) – Michael Michalko
  • The Cement Garden – Ian McEwan
  • Blank Slate: The Modern Denial of Human Nature – Steven Pinker
  • The Big Picture: Reflections on Science, Humanity, and a Quickly Changing Planet – David Suzuki, David Taylor
  • A Short History of Modern Philosophy: From Descartes to Wittgenstein – Roger Scruton
  • Spook Country – William Gibson
  • Secret History – Donna Tartt
  • From Abacus to Zeus: A Handbook of Art History – James Smith Pierce
  • Geek Love: A Novel – Katherine Dunn
  • Flight Behavior: A Novel – Barbara Kingsolver
  • Made to Stick: Why Some Ideas Take Hold and Others Come Unstuck – Chip Heath
  • Mere Christianity – C. S. Lewis
  • Power of Myth – Joseph Campbell
  • Path of Least Resistance: Learning to Become the Creative Force in Your Own Life – Robert Fritz
  • House – Tracy Kidder
  • How the Mind Works – Steven Pinker
  • Propeller Island – Jules Verne
  • Whew, thats a lot of books. I think if I tackle half of these plus a few dozen others I find at the library as I go, I will be quite happy. If you would like to follow along with my progress, I started using Libib to track this stuff: