Amazing or Amazon Web Services (AWS)?

Working with Amazon Web Services (AWS) is actually quite amazing.  It is not just a hosting platform – which is all I initially needed when I launched marrspoints.com.  You know, the mundane, standard bit of Java, Tomcat and MySQL hosting.  After launching the FinTech start-up, TheSubtractor.com, with Scott Scazafavo last year, it has been a fantastic journey into the many application services that they offer as well.

It is so nice that the Route 53 DNS service integrates so seamlessly with their SSL certificates and elastic load balancers.  It makes certificates dead simple and painless.   It feels like the Geico caveman commercials compared to the work of making SSL sites of the past.

Email services are a snap with Simple Email Service (SES).  Standard email client code such as Apache Commons work right out of the box once your account and domain are cleared by Amazon to send.

Email integration is also done so elegantly.  With bounce, complaint, and delivery notifications done in their Simple Notification Service (SNS) via JSON messages.  The costs are crazy low if you do not mind making the glue to your applications and the services.  Even text messaging is super cheap when you move to add texting in addition to email.

Need a CDN?  No problem, start with some S3 buckets as the backing store, put AWS CloudFront in front of it, add in an SSL cert for the secure pages, and your are up and running in minutes!  Again, dead simple to implement.  Want to make the image management of your CDN native to your in-house developed tool set (versus using the AWS S3 console to upload files)?  Use the AWS SDK in the language of your choice to list the files and upload new ones.  It’s a little sad the sub-directories are not real in S3, but with some code you can fake those pretty well.  You might want to break out only the services you need in the SDK, as the Java version all up is over 100MB!

Need to do some AI or Machine Learning?  Amazon has some amazing services that allow you to spin up and only use the compute when you need it.  You do not need to build out a Hadoop cluster on a bunch of ec2 nodes – they take care of all of that for you.  You supply the PySpark or MapReduce code on top of their Elastic Map Reduce (EMR) services.  They are making so many inroads to Big Data as a service that it really lowers the bar of entry for companies to innovate, test and learn with their data strategies.

For a number of reasons, I learned to move to Ubuntu 16.04 for my main version of Linux on AWS.  Too many 3rd party packages were not available in AMI 2 from AWS.  Things like Zabbbix had to be hacked and AMI 2 was way behind on the version of Tomcat in the repo.   Issues with the MySQL client libraries also come to mind.  Just too much fiddling to get a new node to work with my standard development scheme when they work perfectly from the repos with Ubuntu 16.04 (minus the Oracle version of Java of course).

I am sure I am leaving off dozens of items I found along the way of working 100% in AWS for our production site at TheSubtractor.com.  It has been a really enjoyable journey.  So enjoyable I have not written a blog post since July!  We are no longer in Beta and open to all for registration.  I could not be more happy with AWS – it does what it says it does, and it is super easy to implement and integrate with all of its other services.

Ted Cahall

Intel NUCs make a perfect home cluster

Getting my latest NUC

I am pretty psyched to get my latest Intel NUC.  The NUC7i7DNKE has an 8th generation Intel® Core™ i7 vPro™ 4.2 GHz “Turbo”, quad core processor with 32GB of DDR4 2400 MHz RAM and a 1TGB SSD drive.  Not to mention built in 4K UHD video with HDMI ports and USB 3.0.

My home data center NUC cluster
My home data center NUC cluster

I will use this as my main development machine.  It is crazy that I tend to run out of RAM on my 16GB machines running Ubuntu.

This will be my 9th NUC.  Maybe I am a little too in love with these things.  They make great clusters for home research and development on distributed technologies such as Cassandra and Hadoop.  I have three nodes running Cassandra and Hadoop today – and am looking to add a 4th node when I free up my current development machine NUC.

Quiet, Low Power, great for clustering!

They are whisper quiet and use very low power.  There are 5 in a stack sitting on my desk next to me as I write this, and they make less noise than a single standard PC.  In fact, they seem to make no noise at all.

I also run Windows 10 on one as a home theater type of PC connected to a Samsung UHD TV via HDMI.  These NUCs are awesome.  I gave my old i3 core media NUC to my younger brother as a gift.

Here is an old picture of my early stack of NUCs.  They are each 4″ x 4″.

Ted Cahall