Blog

reading list

Highlights from my Reading List – Week 36

Articles

  1. Exploring Cognitive Engagement in Amazon’s Six Page Narratives – Matthew Tippett
    System 2 thinking (in Kanheman taxonomy) as a design feature when writing memos. 

  2. The Asian century is set to begin – Financial Times
    Some data viz that demonstrates the shifting of polarity in geopolitics.

  3. My Unconventional Path to Building the Future of Supply Chain – Shastri Mahadeo
    The story of how Union Crate, a supply chain startup came to be.

  4. Lessons from Stripe – Mark McGranaghan
    Mark reflects on his three years at Stripe and how they’ve built a distinctive culture based on optimism and ambition.

  5. How Amazon Adapted Its Business Model to India – HBR
    “When Amazon decided to enter the Indian e-commerce market, it was clear from the outset that something would have to give. That something was the very business model that had made Amazon an internet powerhouse in the U.S.” 

    A glimpse into Amazon’s strategy in India. 


  6. What Is Signaling? – Robin Hanson
    What counts as signalling and what doesn’t? 
    “More generally I call a message “signaling” if it has these features:
    It is not sent mainly via the literal meanings of words said.
    It is not easily or soon verifiable.
    It is mainly about the senders’ personal features, perhaps via association with groups.
    It is about sender “quality” dimensions where more is better, so senders want others to believe quality is as high as possible, while others want to assess more accurately. Such qualities are not just unitary, but can include degrees of loyalty to particular allies.”

  7. Cellular Automata (The Nature of Code) – Daniel Shiffman 
    A long, introductory read on cellular automata. 
  8. Life Capital – Erik Torenberg
    A Deep Dive into the Past, Present, & Future of Income Share Agreements
  9. Why Amazon Has No Profits (And Why It Works) – Benedict Evans
    Breaking down Amazon’s capital allocation strategy and growth plans. 
  10. The Velocity of Ideas and the Information Economy – Steve Cheney
    Steve explains why an abundance of information leads to poor discovery. 

 

Uncategorized

Lambda School: A Case Study on How to Scale Effective Learning

“Can we devise teaching-learning conditions that will enable the majority of students under group instruction to attain levels of achievement that can at present be reached only under good tutoring conditions?”

This is the 2 sigma problem as defined by Benjamin Bloom, an educational psychologist who pioneered mastery learning.

This article is inspired from a Tweetstorm I did six months ago. You can find the original thread here.

One year ago, I found Lambda School on Twitter. While researching them online, I came across a thread from Austen talking about introducing Mastery Based Progression to their cohorts. This led me to look into Bloom’s two sigma problem, why it wasn’t solved yet and how technology can help scale effective learning.

Bloom’s 2 sigma Problem

Benjamin Bloom’s research found that one on one tutoring could help achieve 2 sigma improvement over conventional teaching. His objective was to find practical, yet scalable methods of attaining two sigma improvement since tutoring wasn’t scalable in its present form as defined by physical constraints.

A brief overview of the problem.

bloom

Why is Bloom’s 2 sigma problem important?

Effectiveness of one on one tutoring indicates that aptitude achievement correlation can be reduced from 0.6 to 0.25, i.e. effective learning levels the playing field for all those who wish to learn irrespective of skill level. It would represent an order of magnitude higher impact as compared to traditional methods.

This is how the achievement distribution looks like for the three cases mentioned in the overview.

Do0_JrrXUAADvsY

Finding Solutions for Bloom’s 2 sigma problem

Looking for solutions to the 2 sigma problem is a four step process.

  1. Look at the entire teaching/learning process
  2. Break down into mutually independent pieces
  3. Find intervention variables and their effect sizes
  4. Combine variables to achieve intended two sigma improvement

Bloom used this approach to look for viable solutions.

bloom2.PNG

The most important variables with high effect sizes were: Feedback and corrective measures, Cooperative learning, Classroom participation, Tutorial instruction Reinforcement and Graded homework. A combination of these variables enabled by the introduction of suitable tech seemed like a viable solution.

The Evolution of Ed Tech

Education is one of the toughest markets to crack for startups. Regulation, long sales cycles, fragmented markets and illegibility to technology (until now) make ed tech startups difficult to scale and largely outside the purview of venture funding. Until now.

Traditional education was ripe for disruption by MOOCs but this failed miserably. Only the most motivated students could finish courses and completion rates were less than 5%. Asynchronous, do it yourself modes of learning are not effective in their current form. This quote describes why: “The big MOOCs mostly employed smooth-functioning but basic video recording of lectures, multiple-choice quizzes, and unruly discussion forums. They were big, but they did not break new ground in pedagogy.” A richer discussion on the topic can be found here.

Given the advances in tech enabled solutions for various aspects of the learning process such as lesson delivery, feedback mechanisms, ability to replay videos, instant communication, TA support via Zoom/Slack, many people predicted the end of the search for a solution to the 2 sigma problem. They were proven incorrect. Instead of the predicted need of fewer teachers, we require more to achieve 2 sigma improvement with the following caveat: teachers should utilized in an effective manner using tech as leverage.

It takes a few iterations before people find a way to execute correctly. The same pattern can be seen here. The most common startups in ed tech focused on getting scale and distribution with little consideration for summative achievement. Getting courses online wasn’t enough. This article looks at the bastardization of the 2 sigma problem. The following quote summarizes the mistake succinctly:

“So here’s where the bastardization comes in to play.  I think that there has been so much focus on the tutoring part, that we’ve lost sight of the learning part.  “Will it scale?” is arguably the most important question that an ed tech investor will ask.  That’s great, and that’s an absolutely justifiable question.  But if you read through Bloom’s paper, there are two parts to the question.  Will it scale, AND, will it improve learning levels over the conventional baseline.  Like with many “shiny object” technologies, we tend to focus on the scale part and gloss over the improvement part (or worse yet, just “assume” the learning will happen).”

Lambda School’s Foray into Ed Tech

A brief description of Lambda School for the uninitiated: A live, fully online school that trains people to become software engineers, data scientists and designers which is free until you get a job. Instead, students pay a percentage of their income each year after they’re employed, the maximum of which is capped at $30k.

A combination of multiple factors enabled Lambda school’s success at finding a product-market fit: rising  student debt and default rates, reliable communication tools like Slack and Zoom and an innovative business model based on Income Share Agreements to name a few.

Lambda managed to crack the problem by innovating on pedagogy and identifying an arbitrage opportunity in the labor market (supply demand gap for CS grads to start with). The latter is important because it allows lambda to do what they do. Traditional education has no skin in the game and perverse incentives. Lambda is forced to act in the students best interests because it has aligned incentives. It earns only when students get high paying jobs.

A few reasons why lambda school works so well:
1. Small class sizes – each cohort is divided into sections with 8-10 students.
2. World class teachers – practitioners who devised OG curricula at places like Apple
3. Cooperative learning – group learning, capstone projects and build weeks
4. Hungry students – gritty students who will do the work (full time for 9 months)
5. Mastery learning – repeat parts of the course until students demonstrate mastery
6. Extensive TA support – feedback and corrective measures

These variables have high effect sizes as indicated by Bloom’s research. This innovation on the pedagogy front is the moat and is scalable across verticals. More from Caleb, VP of Learning at Lambda here and here. By reverse engineering the job market requirements and using their pedagogical infrastructure Lambda can provide effective education in almost any domain legible to online learning.

Lambda School works. The outcomes and demand are proof. The Series B they raised is going to help them scale. On a recent podcast, Austen mentioned their plans to have 3000 students in 2019 and a few cohorts in the EU. Scaling to millions will be an operational challenge and it will be interesting to see how Lambda tackles this.

Thanks to Anisha, Leon, Vidy and Vijay for helping out!
reading list

Highlights from my Reading List – Week 35

Articles

  1. You and Your Research – Dr. Richard W. Hamming
    A classic that I only recently came across. Talks about how to do good research but is more broadly applicable. 

  2. We Are All Architects Now – Venkatesh Rao
    Architects and engineering mid life crashes.

  3. Swings are Free – How It Actually Works
    The Lambda squad on taking swings.

  4. On How to be Discovered – Steve Cheney
    Writing online is the best way to be discovered.
  5. Constructions in Magical Thinking – Venkatesh Rao
    Ribbonfarm is moving towards a new phase; vgr explains. 

  6. A Big Little Idea Called Legibility – Venkatesh Rao
    One of the most important essays I’ve read on the idea of legibility. 

  7. Does Culture Eat Strategy for Lunch? – Venkatesh Rao
    Vgr on culture and strategy.
  8. Amazon gets an edge with its secret squad of PhD economists – CNN
    Economists at Amazon.
 
reading list

Highlights from my Reading List – Week 34

Articles

  1. Pretending to Care, Pretending to Agree – Venkatesh Rao
    Communities, masks and human stress-strain curves.

  2. The Exercise of Authoritah – Venkatesh Rao
    Collectors and connectors. 

  3. What We Learned from Hiring at Homebrew – Satya Patel
    Satya reflects on the hiring process at Homebrew. 

  4. Ergodicity – via Nassim Taleb
    Ergodicity explained using the difference between ensemble and time probability. 
  5. The Amazing, Shrinking Org Chart – Venkatesh Rao
    Vgr on corporate structures, org charts and how organizations respond to uncertainty. 

  6. The Three-Leaps-of-Faith Rule – Venkatesh Rao
    Originally written for PhD students, the three-leaps-of-faith rule has wide applicability in many domains.