JAW Speak

Jonathan Andrew Wolter

Archive for August, 2009

Movie Review: Bigger Stronger Faster. I ask how far to go for better performance?

with one comment

Reading time: 5 – 8 minutes

Watching a movie that entertains is fun, one that teaches benefits you tomorrow, and ones that make you think in a new way are the best of all. I have a friend who says periodically it’s time to go to a conference, “in order to introduce randomness into the system.” Shake things up. Movies in “Cerebral” category in Netflix are a new way I found to do this. My hope is for comments and further recommendations of thought-worthy movies.

Bigger Stronger Faster is a documentary. bigger stronger fasterTechnically it’s about steroids in American culture, but it also raises the clear lack of consistency we treat other performance enhancers. The director Christopher Bell examines his brothers as they use steroids.

What is an ethical and responsible limit to how far you are willing to go for success? Is it okay to wake up in the morning and say you are destined for greatness – that somehow you were born to give something to the world? (And how far will you then go?) Is it okay to just become a normal, average, person?

  • When Tiger Woods had laser eye correction to 20/15 vision, was that an unethical performance enhancement?
  • How about professional musicians taking beta blockers to eliminate anxiety before performances and auditions?
  • Athletes’ are dependent on cortisone shots (a legal steroid), yet should those be held equal to anabolic steroids?
  • Red blood cell count can be increased by doping, taking EPO (details), high altitude training, or sleeping in an altitude chamber. Two options are illegal, two are legal. Should the end result (higher than natural RBC’s) be the determiner of ethics, rather than the mechanism used to reach it?
  • The US Air Force gives fighter pilots speed (amphetamine) to perform better, is that a rational decision?
  • He interviews a member of the Olympic Doping Committee and is told that routinely US Athletes are flagged for failing drug tests, but still allowed to compete.
  • Attending a Chiropractor Anti-aging specialist Chris is able to say he suspects a hormone deficiency which leads to tests and results where no “healthy” range has been set enables him to get an Human Growth Hormone prescription – legally.
  • Students are interviewed in how easy it is to get Adderoll (just tell your doctor you have trouble focusing, or have it passed around from friends). Are these and other “study drugs” (long but really interesting article) worth it? (Or, should everyone be taking them?)

I’m not ready to jump on the film’s open skepticism of “are steroids actually a health risk?” I don’t think they are naturally necessary and a cautious approach to my health comes intuitively. They cross my line of fair competition. Throughout sports and recreational fitness I was never tempted to try them. But maybe that was just because I wasn’t/didn’t want to become good enough to compete at the highest level?

But how far will we go for performance outside of sports? If you could close 70% more sales by taking “Synthesized Aquatic Maltose” (which I just invented), would you take it? Health Supplements in the US are not regulated to be proven healthy, the FDA has the job of proving them unhealthy.

Under the Dietary Supplement Health and Education Act of 1994 (DSHEA), the dietary supplement manufacturer is responsible for ensuring that a dietary supplement is safe before it is marketed. FDA is responsible for taking action against any unsafe dietary supplement product after it reaches the market. Generally, manufacturers do not need to register their products with FDA nor get FDA approval before producing or selling dietary supplements.FDA on DSHEA

Therefore I could start selling this new supplement and require no doctors or nutritionists to even look at what my customers would start to ingest. Chris actually does this. Entertainingly, he picks up a few illegal day laborers, and invents a product and fills pills with his “proprietary blend” of powders. He does “before/after” pictures the same day at a photo shoot and can start selling this $40/bottle tonic. (Of course, manufacturing cost are under $5/bottle for him).

There is more, such as how Utah’s third largest economy ($2.5-$4 billion/year) is the health supplement industry (Nice article here about Utah’s supplement industry). Legislation from Utah’s Senator Orrin Hatch made for the passing of DSHEA, and continues to enable those too squeamish for “real steroids” to get something that promises the same benefits.

He goes on to show a breed of cow: Belgium Blue. Through 100 years of natural selection, these cows are deficient in Myostatin, a growth factor that limits muscle growth. Video below gives a peek. Researchers are looking to mimic that for fighting Muscular Distrophy in humans. See more freakish links about this gene mutation in humans, cows, or other animals. Note: Clip below from National Geographic, not from the movie.

Chris goes to say that Americans are all about Bigger Stronger Faster, and it’s un-American to be #2. We even have romanticized the concept, calling things bigger than expected as “Xyz, on steroids.” We must win, and we must win better than we previously won.

Written by Jonathan

August 30th, 2009 at 4:25 am

Posted in Movie/Book Reviews

Large Web App Architecture: Yes to Thicker Stack on One Hardware Node, No to Beautiful “Redundant” Spiderwebs

with 4 comments

Reading time: 4 – 7 minutes

My last client our team worked with had a large ecommerce operation. Yearly revenue in the new site is in the high single digit billions of dollars. This necessitates extremely high availability. I will draw an initially favorable looking configuration for this high availability (“beautiful spiderwebs”), but then tear it apart and suggest an alternative (“Thicker Stack on One Hardware”).

1. “Beautiful Spiderwebs” – Often Not Recommended

Here’s one common way people could implement high availability. Notice how there are always multiple routes available for servicing a request. If one BIG IP goes down, there is another to help. And this could be doubled with multiple data centers, failed over with DNS.

The visible redundancy and complexity in one diagram may be appealing. One can run through scenarios in order to make sure that yes, we can actually survive any failure and the ecommerce will not stop.

not recommended spiderweb tiers

So then what could make this my Not Recommended option?

2. Martin’s Reminder how to Think About Nodes

Fowler reminded us in Patterns of Enterprise Application Architecture how to look at distribution and tiers. For some reason people keep wanting to have certain “machines running certain services” and just make a few service calls to stitch up all the services you need. If you’re concerned about performance, though, you’re a looking for punishment. Remote calls are several orders of magnitude greater than in process, or calls within the same machine. And this architectural preference is rarely necessary.

One might lead to the first design with the logic of: “We can run each component on a separate box. If one component gets too busy we add extra boxes for it so we can load-balance our app.” Is that a good idea?

fowler distributed objects not recommended

The above is not recommended:

A procedure call between two separate processes is orders of magnitude slower [than in-process]. Make that a process running on another machine and you can add another order of magnitude or two, depending on the network topography involved. [PoEAA Ch 7]

This leads into his First Law of Distributed Object Design: Don’t distribute your objects!

The solution?

Put all the classes into a single process and then run multiple copies of that process on the various nodes. That way each process uses local calls to get the job done and thus does things faster. You can also use fine- grained interfaces for all the classes within the process and thus get better maintainability with a simpler programming model. [PoEAA Ch 7]

fowler clustered application recommended

3. “Strive for Thicker Stack on One Hardware Node” – Recommended

Observe the recommended approach below. There is still an external load balancer, but after a request is routed to an Apache/Nginx/etc front end, you’re all on one* machine.

If one tier fails on a node, pull the whole node out from rotation. Replace it. And re-enter it in the mix.

Your companies teams have worked together to be able to deploy modular services. So when your ecommerce site needs a merchant gateway processing service, you can include that (library or binary) and run it locally on your node, making a call through to it as needed.

Services are also simpler to deploy, upgrade and monitor as there are fewer processes and fewer differently-configured machines.

recommended thicker nodes tiers

(* I understand there may be the occasional exception for remote calls that need to be made to other machines. Possibly databases, mcached obviously third party hosted services, but the point is most everything else need not be remote.)

4. But, Practically Speaking How Far Do We Go?

A caveat first: these benefits get pronounced as you have more and more nodes. (And thus, more and more complex of spiderwebs of unnecessary failover).

Should there be a database server running on each node? Probably not at first. There is a maintenance associated with that. But after sharding your database and running with replication, why not? This way if a node fails, you simply pull it out and replace it with a functioning one.

5. Checklist of Takeaway Lessons

  1. Keep it local. Local calls orders of magnitude faster than remote calls.
  2. Make services modular so they don’t need to be remote, yet still have all the organizational benefits of separate teams.
  3. Simplicity in node-level-redundancy is preferred over tier-level-redundancy.

Often, people think of high availability with terms such as the following: Round Robin, Load Balancing, and Failover. What do you think of? Leave a comment below with how you meet the trade-offs of designing for HA as well as architectural decisions of low latency.

Written by Jonathan

August 19th, 2009 at 12:55 am

Posted in architecture, code, java

Tagged with ,

What is grad school for a consultant (or practicing software engineer)

with 6 comments

Reading time: 2 – 4 minutes

Does it make sense for practicing software engineer to go to grad school? Technologies change rapidly, and so do our clients and projects. So we have constant opportunities to learn and most importantly to apply ourselves in building production systems. Bonus: putting production systems live into the world teaches a lot more than a good grade in an exam.

I once remember in undergrad my friend getting an offer at Microsoft, but he was torn on going to grad school. Basically his sponsor there told him graduate degrees aren’t worth much, and I’d go so far as to suggest that they can be less useful than varied and interesting real project work.

“Sure, go to cs grad school if it makes your ego feel better, but don’t do it for your career.”

Oh?, my friend said, and if my memory serves me, he went on to grad school.

Everyone’s situation is different. However this is something I’ve struggled with for the last several years: does it make sense to step back and do research, and to do computer science-y things instead of day to day project delivery? Maybe. Maybe not. I’ve more or less made my decision for now. But every few months the nagging urge comes back.

I found these posts helpful for framing my decisions.

  1. http://jxyzabc.blogspot.com/2008/08/cs-grad-school-part-1-deciding-to-apply.html (the whole multi-part series, actually)
  2. http://www.stanford.edu/~pgbovine/grad-school-app-tips.htm – says how it is all about research, and really Ph.D. applications are a job application, where you are applying to do research, rather than a place to be taught. (MBA, Med School, and other higher education avenues are a place you pay to be taught.) Masters degrees are different, but generally still have the same theme “you’re getting paid, so you better love (your) research.”

Update: Mark Needham commented about two pointed posts from Penelope Trunk’s blog. Worth reading.

  1. http://blog.penelopetrunk.com/2008/06/18/seven-reasons-why-graduate-school-is-outdated
    Grad School isn’t needed, and it causes more harm than good. Too much focus. Careers change too much. Experience trumps prolonged childlike academic sheltering. (Note: My opinion isn’t so harsh. Your grad school might not apply, it depends on what you want to do).
  2. http://blog.penelopetrunk.com/2009/02/03/dont-try-to-dodge-the-recession-with-grad-school/
    She prefers learning by doing, and suggests a feeling of “being lost” is actually helpful for growing.

Written by Jonathan

August 10th, 2009 at 7:21 pm

Posted in career, code

Tagged with ,

Merging pdf’s on Mac OS X from a non-duplex scanner

with one comment

Reading time: 2 – 3 minutes

Goal: scan in hundreds of duplex documents in a non-duplex scanner and combine into 1 pdf in automated way. Status: it was harder than it should have been, and not that automated, but this works.

Scan in the papers as pdf’s from your paper-feed equipped scanner. Scan them right side up, then flip over and scan the other sides. The two pdf’s will contain pages: 1, 3, 5… and 2, 4, 6…

Reverse the even pages.

#!/usr/bin/ruby
 
if __FILE__ == $0
  puts "Run this on ubuntu or somewhere that pdftk is easy to be had. (which isn't os x)"
 
  if ARGV.length != 1
    puts "Syntax: #{__FILE__} pdf_to_reverse.pdf"
    exit
  end
 
  pdf = ARGV[0]
  reversed_pdf = pdf.gsub(/\.pdf/i, "_reversed.pdf")
 
  page_count = `pdfinfo #{pdf} | grep Pages`.scan(/\d+/)
 
  `pdftk #{pdf} cat #{page_count}-#{1} output #{reversed_pdf}`
end

Lastly, combine the two pdf’s, shuffling every other page, starting with the odds. Note it has some dependencies on pdftk and pdfinfo for the reversing (which are excruciatingly difficult to install on os x), and os x (for the merging).

#!/usr/bin/ruby
 
if __FILE__ == $0
  puts "Run this on os x to shuffle two pdf's, where the
        even pages are already reversed (reverse them with other script)"
 
  if ARGV.length != 3
    puts "Syntax: #{__FILE__} odds.pdf reversed_evens.pdf output.pdf"
    exit
  end
 
  odds_pdf = ARGV[0]
  reversed_evens_pdf = ARGV[1]
  output_pdf = ARGV[2]
 
  # obviously, only works on os x.  I didn't see an easy way to combine pdf's
  # in pdftk or other tools I searched for
  `python '/System/Library/Automator/Combine PDF Pages.action/Contents/Resources/join.py' --output '#{output_pdf}' --shuffle '#{odds_pdf}' '#{reversed_evens_pdf}'`
end

References:

  • pdftk – pdf toolkit, I could have installed with ports install pdftk, but that has a very long build dependency on gcj.
  • Another technique which would work if you didn’t need to reverse pages, using automator. And without automator (like I do with a script directly).

Written by Jonathan

August 5th, 2009 at 8:41 am

Posted in automation