Code Boot Camps Fail Obama As Unaudited Stats Cast Doubt On Success Rates

SAN FRANCISCO — Visit the website of any top coding school and you'll find some eye-popping numbers: Some promise to turn people into software engineers in as little as 12 weeks and most claim job placement rates of 97, 98 or 99 percent within months of graduation.

But those claims are largely unaudited by third parties and based on differing standards, which makes it difficult to compare the effectiveness of programs. For example, schools tend to omit the completion rates for these programs. New York City's General Assembly, for example, boasts a 99 percent job placement rate but does not disclose its completion rate. Denver's Turing School, meanwhile, touts a placement rate of 98 percent but has a drastically lower completion rate, of 71 percent.

Additionally, there is no standard for what constitutes a hire or what time frame should be used to measure success. At App Academy, a 12-week program based in San Francisco, only a full-time software developer position counts, but its 98 percent placement rate is measured over a yearlong period. Miami-based Wyncode, meanwhile, claims a more modest 90 percent placement rating that includes internships and part-time positions but is measured over just three months.

“Oftentimes, boot camps will make these numbers up,” said Jonathan Lau, co-founder of SwitchUp, an independent directory of boot-camp reviews and rankings. “They’ll exclude students who they think were bad and mark them as a failed student and do all sorts of weird things that bump up their numbers and make it look better. With a third-party auditor, you can’t really do that.”

Nearly a year ago, these schools along with six others made a pledge to reform. The schools, part of the New Economy Skills Training Association, or Nesta, sent a letter to President Barack Obama last March expressing their commitment to release standardized, annual audited outcomes reports. It was part of a "Tech Hire" White House initiative to get more Americans trained for tech jobs.

Yet almost 12 months later, only one of the 10 schools has delivered on its promise to the White House, and it’s been several months since the Nesta schools have come together for a meeting, the organization along with several of the schools told International Business Times.

"’Disappointed’ might be a little too strong, but I'm underwhelmed that we haven't lived up to that just yet," said Turing School Executive Director Jeff Casimir, referring to the letter to Obama, sent on March 10, 2015. "With Turing, transparency is at the core of what we do ... and that gap needs to be fixed.”

Besides Turing School, others that made the commitment included App Academy, Dev Bootcamp, Flatiron School, General Assembly, Galvanize, Hackbright Academy, Hack Reactor, MakerSquare and Wyncode — some of the most recognizable brands among coding boot camps. New York City's Flatiron is the only school to have released third-party verified and audited results.

<iframe src="//datawrapper.dwcdn.net/RJyaQ/1/" frameborder="0" allowtransparency="true" allowfullscreen="allowfullscreen" webkitallowfullscreen="webkitallowfullscreen" mozallowfullscreen="mozallowfullscreen" oallowfullscreen="oallowfullscreen" msallowfullscreen="msallowfullscreen" width="600" height="520"></iframe>

The rest remain at various stages of the process. A handful — Hackbright, Hack Reactor, MakerSquare and Turing — are aiming to release audited results before the end of the year. App Academy, Dev Bootcamp and General Assembly say they plan to deliver on their promise but have no specific timeline. And Wyncode said it already provides its outcomes to the Florida Department of Education but remains committed to releasing audited results as well, although no timeline is in place.

Meanwhile, Galvanize, which has eight campuses across the U.S., has instead turned its focus toward working with state and federal agencies to deliver quality assurance to its students. “Even if verified by outside entities, independent reporting will be of limited value until participants use common definitions and measurements,” the school said in a statement.

While this market of predominantly for-profit, venture-backed businesses has only been around about four years, it is beginning to see exponential growth. Already, there are 69 full-time, in-person programs across the U.S., up from 43 in 2014, and in 2016, the industry is expected to graduate approximately 28,000 students, up 100 percent from 14,000 graduates in 2015, according to an analysis by SwitchUp. The average tuition at these schools has also increased, from about $10,000 in 2014 to $11,000 in 2015, SwitchUp said.

Third-party verified outcomes are key to ensuring the integrity of the fast-growing coding boot-camp market, experts said. That is crucial to maintaining the public’s trust and avoiding government intervention, as has occurred with other for-profit schools. Just last month, the Federal Trade Commission sued DeVry University for deceptive marketing.

Coding boot camps would like to avoid that fate, several school representatives said.

“There’s very little barriers to entry in the market, and there’s a high risk of low-quality providers rushing in. It’s an easy market to get into,” said Sharon Weinbar, CEO of Hackbright Academy in San Francisco. “We want prospective students to feel confident when they choose to apply.”

Yet disagreements among Nesta members have delayed efforts to release audited results. This discord stems from schools wanting to use standards that will best benefit their own numbers, various schools said.

“[For] a couple of schools that are a lot larger that are a part of Nesta ...their numbers may not look as impressive if it was done in a certain way,” said Wyncode Co-Founder Juha Mikkola. “I think that may have slowed down the process a little bit in terms of agreeing to what standards outcome we would all come around the table to report.”

The critical questions? How to define a job placement and which students should not count as dropouts. “With the completion rate, how do you treat someone who leaves the program for a medical reason?” said Ruan Pethiyagoda, chief strategy officer of Reactor Core, parent company of San Francisco's Hack Reactor and MakerSquare, which has three campuses around the country. Pethiyagoda said there is even more disagreement when it comes to the job placement rate. “What do you do with people who we [the school] have hired? I’m one of them.”

This division has led to a stalemate in collaboration at Nesta. According to many of the participating schools, it has been “months” since the trade association has come together, even by phone, for a meeting of any significance. Dissent has also begun. Galvanize CEO Jim Deters on Tuesday described his school as having “formerly participated in” Nesta.

Nesta Chairman Ryan Orban on Tuesday told IBT that the group has been on a brief hiatus and "is not a standards-setting body."

“Member schools have focused the last several months on coming into compliance with their local state regulations, which require reporting and accurate representations of outcomes,” said Orban, who is also vice president of product and strategy at Galvanize. “Nesta is committed to advocating for high-quality immersive education and is currently reorganizing to support that goal.”

Many coding boot camps would like a standardized reporting consensus before having their results audited, but putting out some verified data is better than doing nothing, said Flatiron School CEO Adam Enbar.

“Saying we can’t agree so were gonna do nothing is an absurd excuse,” Enbar said. “It’s like saying ‘Well, I didn’t know the answer to the first question on the test, so I didn’t try on the rest of it.’”

At Flatiron School, the only Nesta boot camp to release an audited report, there is a very loose but transparent definition for job placement that includes an overall figure as well as a breakdown of which positions were full-time, apprenticeships, contractors, freelance and unpaid. Altogether, Flatiron boasts an overall job placement rating of 98 percent for 2015, but its placement rating falls to 58 percent when only counting full-time salaried positions.

“People come here because they want to get a job as a software engineer,” Enbar said. “It's important that we are not misleading students into believing they're going to get something they're not going to get."

Though maintaining up-to-date job placement statistics is a challenge for schools -- Hack Reactor, for example, has a team of five individuals who contacts alums and another two staffers who handle the collected data, Pethiyagoda said -- having that data audited should not be.

Flatiron School’s Enbar said conducting a third-party audit is a process that requires less than $5,000. The process begins by sending a spreadsheet of the school's data to the auditors who then comb through it and at random call up about 30 percent of the students on the list. The auditors' require a 100 percent response rate and then verify every bit of information, ensuring that the spreadsheet's data is 100 percent lined up with the alumni's answers.

“If you’ve been around more than a year, the only way you can’t afford it is because you’re paying yourself a lot of money or you’re just using that money to invest in growth, open more locations and enroll more students,” Enbar said. “Five-thousand dollars is a fraction of one student’s tuition out of hundreds of students that schools enroll every year.”

The other nine schools who wrote to Obama still have one month left to deliver on the promise of annual, verified outcomes, but it will be disappointing if the schools renege on their letter, Enbar said.

"There are amazing schools out there, but it's hard to know because without transparency, we're all taking everybody's word for it,” he said.