1. The Mythology of Big Data (PDF) -- slides from a Strata keynote by Mark R. Madsen. A lovely explanation of the social impediments to the rational use of data. (via Hamish MacEwan)
2. Scamworld -- amazing deconstruction of the online "get rich quick" scam business. (via Andy Baio)
3. Ceres: Solving Complex Problems with Computing Muscle -- Johnny Lee Chung explains the (computer vision) uses of the open source Ceres Non-Linear Least Squares Solver library from Google.
4. How to Start a Think Tank (Guardian) -- The answer to the looming crisis of legitimacy we're facing is greater openness - not just regarding who met who at what Christmas party, but on the substance of policy. The best way to re-engage people in politics is to change how politics works - in the case of our project, to develop a more direct way for the people who use and provide public and voluntary services to create better social policy. Hear, hear. People seize on the little stuff because you haven't given them a way to focus something big with you.

JavaScript keeps advancing by leaps and bounds, but is it powerful enough yet? Is the Web ready to take on all the challenges we throw at it?

I talked with Seth Ladd, a web engineer and Chrome Developer Advocate at Google who's working on Dart, but still, I'm happy to say, interested in JavaScript itself. He's been working with larger projects and larger teams figuring out how to build bigger, faster, and more complex applications than most of us care to dream about.

Seth's constant push - "we can do better" - takes a hard look at where we are today with web programming, acknowledging decades of improvement but looking hard for the next best thing.

Highlights from the full video interview include:

• Speed - is JavaScript fast enough yet? [Discussed at the 2:12 mark]
• 60 frames per second - can the browser look that smooth? [Discussed at the 3:21 mark]
• Dart - Structure, tooling, and reaching both JavaScript and C++ programmers [Discussed at the 6:27 mark]
• "Dart compiles to modern JavaScript today" [Discussed at the 9:16 mark]
• "JavaScript is becoming the bytecode of the Web" - many languages compile to JavaScript [Discussed at the 11:16 mark]
• View Source isn't what it used to be - is Github the answer? [Discussed at the 12:07 mark]

You can view the entire conversation in the following video:

Fluent Conference: JavaScript & Beyond — Explore the changing worlds of JavaScript & HTML5 at the O'Reilly Fluent Conference (May 29 - 31 in San Francisco, Calif.).



Save 20% on registration with the code RADAR20

Related:

• What is Dart?
• Subscribe to the free Code podcast through iTunes

This is part of the Velocity Profiles series, which highlights the work and knowledge of web ops and performance experts.

Justin Huff
Software Engineer
PicMonkey
@jjhuff

How did you get into web operations and performance?

Picnik's founders Mike Harrington and Darrin Massena needed someone who knew something about Linux. Darrin and I had known each other for a few years, so my name came up. At the time, I was doing embedded systems work, but ended up moonlighting for Picnik. It wasn't long before I came over full time. I always expected to help them get off the ground and then they'd find a "real sysadmin" to take over. Turns out, I ended up enjoying ops! I was lucky enough to straddle the world between ops and back-end dev. Sound familiar?

What is your most memorable project?

Completing a tight database upgrade at a Starbucks mid-way between Seattle and Portland. "Replicate faster, PLEASE!" Also, in the build-up to Picnik's acquisition by Google, Mike asked me what it would take to handle 10 times our current traffic and to do it in 30 days. We doubled Picnik's hardware, including a complete network overhaul. It went flawlessly and continued to serve Picnik until Google shut it down in April of this year.

What's the toughest problem you've had to solve?

When Flickr launched with Picnik as its photo editor, we started to see really weird behavior causing some Flickr API calls to hang. I spent a good chunk of that day on the phone with John Allspaw and finally identified an issue with how our NAT box was munging TCP timestamps that were interacting badly with Flickr's servers. I learned a couple things: First, both John and I were able to gather highly detailed info (tcpdumps) at key points in our networks (and hosts) — sometimes you just have to go deep; second, it's absolutely imperative that you have good technical contacts with your partners.

What tools and techniques do you rely on most?

Graphs and monitoring are critical. Vim, because I can't figure out Emacs. Automation, because I can't even remember what I had for breakfast.

Who do you follow in the web operations and performance world?

Bryan Berry (@bryanwb) is great. Joe Williams (@williamsjoe) is doing great stuff — and his Twitter profile pic is awesome.

What is your web operations and performance super power?

I think I'm good at building, maintaining, and understanding complete systems. Other engineering disciplines are typically concerned about the details of a single part of a larger system. As web engineers, we have to grok the system, the components, and their interactions ... at 2 AM.

Velocity 2012: Web Operations & Performance — The smartest minds in web operations and performance are coming together for the Velocity Conference, being held June 25-27 in Santa Clara, Calif.

Save 20% on registration with the code RADAR20

Related:

• See more Velocity Profiles

The last couple of days, there's been a fair amount of blogosphere angst over Coding Horror's "Please Don't Learn to Code." Ironically, the best argument for learning to code appeared this morning, when it turned out that Judge William Alsup in the Google case could program, and learned Java in the course of the trial, and wasn't going for Oracle's claim that a short range-checking function was days of work. Alsup recognized immediately (and says he wrote the function hundreds of times during the course of the trial) that it's just a few minutes work for a competent programmer.

The importance of learning to code isn't so that everyone will write code, and bury the world under billions of lines of badly conceived Python, Java, and Ruby. The importance of code is that it's a part of the world we live in. I've had enough of legislators who think the Internet is about tubes, who haven't the slightest idea about legitimate uses for file transfer utilities, and no concept at all about what privacy (and the invasion of privacy) might mean in an online space. I've had enough of patent inspectors who approve patents for which prior art has existed for decades. And I've had enough of judges making rulings after listening to lawyers arguing about technologies they don't understand. Learning to code won't solve these problems, but coding does force engagement with technology on a level other than pure ignorance.

Coding is a part of cultural competence, even if you never do it professionally. Alsup is a modern hero.

Great piles of cash are descending on entrepreneurs who develop health care apps, but that doesn't make it any easier to create a useful one that your audience will adopt. Furthermore, lowered costs and streamlined application development technique let you fashion a working prototype faster than ever, but that also reduces the time you can fumble around looking for a business model. These were some of the insights I got at Spring Fling 2012: Matchpoint Boston, put on by Health 2.0 this week.

This conference was a bit of a grab-bag, including one-on-one meetings between entrepreneurs and their potential funders and customers, keynotes and panels by health care experts, round-table discussions among peers, and lightning-talk demos. I think the hallway track was the most potent part of this conference, and it was probably planned that way. The variety at the conference mirrors the work of Health 2.0 itself, which includes local chapters, challenges, an influential blog, and partnerships with a range of organizations. Overall, I appreciated the chance to get a snapshot of a critical industry searching for ways to make a positive difference in the world while capitalizing on ways to cut down on the blatant waste and mismanagement that bedevil the multi-trillion-dollar health care field.

Let's look, for instance, at the benefits of faster development time. Health IT companies go through fairly standard early stages (idea, prototype, incubator, venture capital funding) but cochairs Indu Subaiya and Matthew Holt showed slides demonstrating that modern techniques can leave companies in the red for less time and accelerate earnings. On the other hand, Jonathan Bush of athenahealth gave a keynote listing bits of advice for company founders and admitting that his own company had made significant errors that required time to recover from. Does the fast pace of modern development leave less room for company heads to make the inevitable mistakes?

I also heard Margaret Laws, director of the California HealthCare Foundation's Innovations Fund, warn that most of the current applications being developed for health care aim to salve common concerns among doctors or patients but don't address what she calls the "crisis points" in health care. Brad Fluegel of Health Evolution Partners observed that, with the flood of new entrepreneurs in health IT, a lot of old ideas are being recycled without adequate attention to why they failed before.

I'm afraid this blog is coming out too negative, focusing on the dour and the dire, but I do believe that health IT needs to acknowledge its risks in order to avoid squandering the money and attention it's getting, and on the positive side to reap the benefits of this incredibly fertile moment of possibilities in health care. Truly, there's a lot to celebrate in health IT as well. Here are some of the fascinating start-ups I saw at the show:

• hellohealth aims at that vast area of health care planning and administration that cries out for efficiency improvements--the area where we could do the most good by cutting costs without cutting back on effective patient care. Presenter Shahid Shah described the company as the intersection of patient management with revenue cycle management. They plan to help physicians manage appointments and follow-ups better, and rationalize the whole patient experience.

  hellohealth will offer portals for patients as well. They're unique, so far as I know, in charging patients for certain features.

• Corey Booker demo'd onPulse, which aims to bring together doctors with groups of patients, and patients with groups of the doctors treating them. For instance, when a doctor finds an online article of interest to diabetics, she can share it with all the patients in her practice suffering from diabetes. onPulse also makes it easier for a doctor to draw in others who are treating the same patient. The information built up about their interactions can be preserved for billing.

  onPulse overlaps in several respects with HealthTap, a doctor-patient site that I've covered several times and for which an onPulse staffer expressed admiration. But HealthTap leaves discussions out in the open, whereas onPulse connects doctors and patients in private.

• HealthPasskey.com is another one of these patient/doctor services with a patient portal. It allows doctors to upload continuity of care documents in the standard CCD format to the patient's site, and supports various services such as making appointments.

  A couple weeks ago I reported a controversy over hospitals' claims that they couldn't share patient records with the patients. Check out the innovative services I've just highlighted here as a context for judging whether the technical and legal challenges for hospitals are really too daunting. I recognize that each of the sites I've described pick off particular pieces of the EHR problem and that opening up the whole kit and kaboodle is a larger task, but these sites still prove that all the capabilities are in place for institutions willing to exploit them.

• GlobalMed has recently released a suitcase-sized box that contains all the tools required to do a standard medical exam. This allows traveling nurse practitioners or other licensed personnel to do a quick check-up at a patient's location without requiring a doctor or a trip to the clinic. Images can also be taken. Everything gets uploaded to a site where a doctor can do an assessment and mark up records later. The suitcase weighs about 30 pounds, rolls on wheels, and costs about $30,000 (price to come down if they start manufacturing in high quantities).

• SwipeSense won Health 2.0's 100 Day Innovation Challenge. They make a simple device that hospital staff can wear on their belts and wipe their hands on. This may not be as good as washing your hands, but takes advantage of people's natural behavior and reduces the chance of infections. It also picks up when someone is using the device and creates reports about compliance. SwipeSense is being tested at the Rush University Medical Center.

• Thryve, one of several apps that helps you track your food intake and make better choices, won the highest audience approval at Thursday's Launch! demos.

• Winner of last weekend's developer challenge was No Sleep Kills, an app that aims to reduce accidents related to sleep deprivation (I need a corresponding app to guard against errors from sleep-deprived blogging). You can enter information on your recent sleep patterns and get back a warning not to drive.

It's worth noting that the last item in that list, No Sleep Kills, draws information from Health and Human Services's Healthy People site. This raises the final issue I want to bring up in regard to the Spring Fling. Sophisticated developers know their work depends heavily on data about public health and on groups of patients. HHS has actually just released another major trove of public health statistics. Our collective knowledge of who needs help, what works, and who best delivers the care would be immensely enhanced if doctors and institutions who currently guard their data would be willing to open it up in aggregate, non-identifiable form. I recently promoted this ideal in coverage of Sage Congress.

In the entirely laudable drive to monetize improvements in health care, I would like the health IT field to choose solutions that open up data rather than keep it proprietary. One of the biggest problems with health care, in this age of big data and incredibly sophisticated statistical tools, is our tragedy of the anti-commons where each institution seeks to gain competitive advantage through hoarding its data. They don't necessarily use their own data in socially beneficial ways, either (they're more interested in ratcheting up opportunities for marketing expensive care). We need collective sources of data in order to make the most of innovation.

Before I came to O'Reilly I was building the "big data and disruptive analytics practice" at a major systems integrator. It was a blast to spend every week talking to customers in different industries who were waking up to the possibilities that technologies like Hadoop offered their businesses. Many of these businesses are going to fundamentally change as they embrace this stuff (or be replaced by those that do). But there's a catch.

Twenty years or so ago large integrators made big business building applications on the then-new relational paradigm. They put in Oracle databases with custom code, wrote PowerBuilder apps on Sybase, and of course lots of businesses rolled their own with VB and SQL Server. It was an era of custom coding where Oracle, Sybase, SQL Server, Informix and etc. were thought of as platforms to build stuff on.

Then the market matured and shifted to package solution implementation. ERP, CRM, …, etc. The big guys focused on integrating again and told their clients there was no ROI in building custom stuff. ROI would come from integrating best-of-breed solutions. Databases became commodity back ends to the applications that were always the real focus.

Now along comes big data, NoSQL, data science, and all that stuff and it seems like we're starting the cycle over again. But this time clients, having been well trained over the last decade or so, aren't having any of that "build it from scratch" mentality. They know that Hadoop and other new technologies can be transformative to their business, but they want it packaged up and solution'ified like they are used to. I heard a lot of "let us know when you have a solution already built or available to buy that does X" in the last year.

Also, lots of the shops that do this stuff at scale are built and staffed around the package implementation model and have shed many of the skills they used to have for custom work. Everything from staffing models to methodologies are oriented toward package installation.

So, here we are with all of this disruptive technology, but we seem to have lost the institutional wherewithal to do anything with it in a lot of large companies. Of course that fact was hard on my numbers. I had a great pipeline of companies with pain to solve, and great technologies to solve it, but too much of the time it was hard to close it without readymade solutions.

Every week I talked to the companies building these new platforms to share leads and talk about their direction. After a while I started cutting them off when they wanted to talk about the features of their next release. I just got to the point where I didn't really care, it just wasn't all that relevant to my customers. I mean, it's important that they are making the platforms more manageable and building bridges to traditional BI, ETL, RDBMS, and the like. But the focus was too much on platforms and tools.

I wanted to know "What are you doing to encourage solution development? Are you staffing a support system for ISVs? What startups and/or established players are you aware of that are building solutions on this platform?" So when I saw this tweet I let out a little yelp. Awesome! The lack of ready-to-install solutions was getting attention, and from Mike Olsen.

Cloudera CEO wants startups to build Hadoop apps. He will connect you to funding. #dataconf

— MJHarkins (@uberjake42) March 21, 2012

You can watch the rest of what Mike Olson said here and you'll find he tells a similar story about the RDBMS historical parallel.

I talked to Mike a few weeks ago to find out what was behind his comment and explore what else they are doing to support solution development. It boils down to what he said — he will help connect you with money — plus a newly launched partner program designed to provide better support to ISVs among others. Also, the continued attention to APIs and tools like Pig and Hive should make it easier for the solution ecosystem to develop. It can only be good for his business to have lots of other companies directly solving business problems, and simply pulling in his platform.

Hortonworks also started a partner program in the fall and I think we'll see a lot more emphasis on this across the space this year. However, at the moment wherever I look (Hortonworks partners, Cloudera Partners, Accel big data portfolio) the focus today remains firmly on platform and tools or partnering with integrators. Tresata, a startup focused on financial risk management, pops up in in a lot of lists as the obvious odd one out — an actual domain-specific solution.

What about other people that could be building solutions? Is it the maturity level of the technology, the lack of penetration of Hadoop etc. into your customer's data centers, or some combination of other factors that is slowing things down?

Of course, during the RDBMS adoption it took a lot of years before the custom era was over and thoroughly replaced by the era of package implementation. The question I'm pondering is whether customer expectations and the pace of technology will make it happen faster this time? Or is the disruptive value of big data going to continue to accrue only to risk-taking early adopters for the foreseeable future?

If you are building a startup based on a solution or application that leverages big data technology, and you aren't being stealthy, I'd love to hear about it in the comments.

Related:

• Series: "Planning for big data"
• Building data startups: Fast, big, and focused
• Hadoop: What it is, how it works, and what it can do
• What is data science?