The Planning Fallacy: Why We Get Time Estimates Wrong

Despite years of experience, I still fall into the same trap as professionals everywhere – that is consistently misjudging how long a task will take. This common ‘miscalculation,’ termed the planning fallacy, refers to the systematic tendency to underestimate task completion times, even when individuals have previously undertaken similar projects. It can be mildly inconvenient if you have promised to help a friend with what seems (at first sight) to be a simple task. However, it is more problematic if you cost your services based on how long you expect a project will take, leading to missed deadlines, budget overruns, and increased stress [1]. Vague commitments and well-intended goals rarely work, effectively they torpedo your ‘best made’ plans, and the larger the task the greater the uncertainty. Facilis descensus Averno, as they say, or effectively, "The road to hell is paved with good intentions."

First described by Kahneman and Tversky in 1979, the planning fallacy persists in high-functioning individuals and expert teams like ours at Niche [2]. In our early days of business, we found like others before us that our underestimations occurred repeatedly even though we had prior experience with similar tasks [3]. Review of our approach found that our issues were arising from a reliance on the ‘inside view,’ focusing narrowly on the specifics of a task while neglecting broader complication [4]. We found that to counteract this, you should adopt an ‘outside view,’ leveraging historical data and comparative benchmarks to build objective estimates. Following our own assessment, we became more serious about how we recorded time utilisation and paid closer attention to task complexity with resource requirements. The inclusion of techniques such as percentage-based buffering and task decomposition into smaller, trackable steps also proved effective in improving estimation accuracy.

Understanding the Planning Fallacy

The planning fallacy arises from optimistic bias and over-reliance on internal scenarios for task completion. We tend to focus on best-case scenarios, often failing to account for task-switching inertia, typical delays, interruptions, and unanticipated complexities [2]. This bias or time blindness is compounded by memory distortions, where past underestimations are forgotten or misattributed. In a seminal study in 1994, university students were asked to estimate how long it would take to complete their honours thesis. The average estimate was 33.9 days; actual completion averaged 55.5 days. Notably, even students who had completed similar tasks in the past remained overly optimistic [3].

Similar effects are observed in corporate settings. A review of over 200 infrastructure projects found that 90% exceeded time estimates, with consistent patterns of resource underestimation [5]. In software development, the Standish Group's CHAOS Report (2020) showed that most IT projects exceed time and cost expectations [6].

Why Experience Doesn’t Cure the Fallacy

We all have an inherent optimism bias, believing that whatever tasks we are planning will proceed smoothly [7]. But we know that there are always obstacles, interruptions, and unforeseen complications. You might expect professionals with years of experience to foresee and overcome such biases. Research shows that experience alone does not inoculate individuals against the planning fallacy. Experts are often overconfident in their abilities and rely more heavily on intuition than on past performance data or evidence-based forecasting [8]. This cognitive blind spot in reflective learning leads to flawed predictions [9].

Numerous studies, including those from the Tufts Center for the Study of Drug Development, have documented the high frequency of change orders and budget underestimation in pharmaceutical outsourcing. Up to 57% of clinical trials experience major protocol amendments, often due to overly optimistic feasibility assessments and sponsor-CRO misalignment, which significantly inflate timelines and costs [10][11][12]. But we are in good company. Software developers consistently underestimated how long it will take to complete programming tasks [13]. For example, the National Programme for IT (NPfIT) launched in 2002 aimed to create a unified, nationwide electronic health records system. Originally expected to cost around £6.4 billion, public accounts later revealed that between £12 billion and £20 billion was spent before the programme was dismantled around 2011, with minimal clinical value delivered [14]. The House of Lords Economic Affairs Committee identified optimism bias and inside-view forecasting as key contributors to the underestimation of cost and timeline [15].

The Outside View: An Antidote to Optimism Bias

One of the most effective means of addressing planning fallacy is the adoption of the "outside view," otherwise known as reference class forecasting [4]. Instead of imagining how a specific task will unfold, the outside view considers the statistical outcomes of similar past tasks. This shift reduces the influence of optimistic scenarios and anchors estimates in historical data. For example, if writing a regulatory submission historically takes a team 8 to 12 weeks, that timeframe should be the starting point for delivery milestones, irrespective of a new project’s unique characteristics.

This strategy is especially effective when combined with project management tools and benchmarking data from previous ‘similar’ projects. For example, at Niche we have 26 years of data records on how long it takes to write CSR’s, protocols and Investigator’s Brochures, etc. This is helpful when clients are pressuring for the lowest possible cost and insisting that "This time will be different." Studies show that when you embrace using the outside view you get significantly more accurate forecasts [4].

Percentage Buffers vs Fixed-Time Additions

A common practice for addressing potential underestimation is to add a fixed amount of time to initial predictions, particularly when clients exhibit high cost sensitivity. However, this method is prone to error compared with percentage-based buffers. Simply adding “a little extra time” will never be enough. Analysis of our own historical data showed that this is because under estimations scale with the length and complexity of the task. That percentage-based buffers (e.g., adding 25% to the estimated time) more accurately accounts for unplanned complications than flat buffers (e.g., simply adding 2 days to your budget) has been known for some time [16].

For instance, a 25% buffer on a 20-day task (resulting in 25 days total) is more adaptive than a 2-day buffer, which could significantly underestimate the time needed for larger or more variable projects. Percentage buffers should be calibrated based on empirical data. Teams should track their average underestimation over time and adjust their standard buffer percentages accordingly [17].

Breaking Projects into Smaller Tasks

Task decomposition is another powerful budgeting technique. Large projects invite greater estimation errors due to their complexity and uncertainty. By breaking projects into smaller, well-defined subtasks, individuals and teams empowers you to generate more accurate time estimates and identify potential bottlenecks earlier. Research in cognitive psychology suggests that people are better at predicting durations of simple, short tasks than complex, extended ones [18]. This is partly due to the reduced variability and fewer dependencies within smaller tasks. This also allows for incremental adjustments as work progresses.

Agile methodologies in software development exemplify this principle. By working in short sprints and estimating the time needed for each task or user story, teams generate more accurate forecasts and can adapt to changes more quickly. Similar approaches are now being adopted in teams like ours working in medical writing, regulatory affairs, and other project-driven domains. Studies in construction and software engineering reveal that projects using Work Breakdown Structures (WBS) experience fewer delays [19].

Overcoming Institutional and Cultural Barriers

Organisational culture can reinforce the planning fallacy. When underestimating timelines is rewarded (e.g., promising faster delivery to secure contracts), teams may internalise unrealistic standards. Over time, this leads to normalised overpromising and underdelivering.

Counteracting this requires institutional change on top of accurate recording of resource utilisation. Transparent post-project reviews, performance metrics linked to estimation accuracy, and leadership support for realistic forecasting are crucial. Tools like Gantt charts, burndown charts, and earned value management can also help visualise progress and identify discrepancies between planned and actual timelines.

Conclusion

Seasoned professionals are vulnerable to the planning fallacy, which leads to systemic underestimation of task durations. This bias stems from inherent cognitive tendencies, exacerbated by overconfidence and organizational pressures.

Adopting an outside view, using percentage-based buffers, and decomposing large projects into smaller steps are evidence-based strategies to improve time estimation. Obviously, this requires that you keep records on your past projects. While experience is valuable, it must be coupled with structured reflection, empirical data, and adaptive project management tools to counteract the persistent optimism of the human mind.

Learning to make more accurate time estimates can significantly reduce stress, enhance the quality of your work, and strengthen your reputation for reliability. Paradoxically, careful planning often results in time saved. When tasks are completed within realistic timeframes, the risk of becoming overwhelmed diminishes, freeing up mental space for deeper, more creative thinking. The next time you catch yourself saying, “This will only take a few minutes,” pause to consider the planning fallacy, our common tendency to underestimate how long tasks will take. A more grounded assessment of complexity will serve you well. You will thank yourself for interrupting your descent into Hell.

References

1. Flyvbjerg B. (2008). "Curbing optimism bias and strategic misrepresentation in planning." Europ Plan Stud 16(1), 3–21.
2. Kahneman D, Tversky A. (1979). Intuitive prediction: Biases and corrective procedures. In S. Makridakis & S. C. Wheelwright (Eds.), Studies in the Management Sciences: Forecasting (Vol. 12, pp. 313–327). North-Holland.
3. Buehler R, et al. (1994). Exploring the "planning fallacy": Why people underestimate their task completion times. J Person Soc Psychol 67(3), 366–381.
4. Kahneman D, Lovallo D. (1993). Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Sci 39(1), 17–31.
5. Flyvbjerg B, et al. (2003). How common and how large are cost overruns in transport infrastructure projects? Transport Rev 23(1), 71–88.
6. Standish Group. (2020). CHAOS Report 2020. Standish Group International.
7. Sharot T. (2011). The optimism bias. Curr Biol 21(23):R941–5.
8. Moore DA, Healy PJ. (2008). The trouble with overconfidence. Psychol Rev 115(2), 502–517.
9. Kahneman D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
10. Getz KA. (2012). Improving protocol design feasibility to drive drug development economics and performance. Clin Investig (Lond) 2(9):855–860.
11. Mollan KR, et al. (2020). Protocol amendments: a rarely reported but increasingly common phenomenon in clinical trials. Contemp Clin Trials Commun 19:100655
12. Tufts Center for the Study of Drug Development. Impact Report Summary: Protocol Amendments – Frequency, Cost, Causes and Impact. Boston: Tufts University; 2016.
13. Halkjelsvik T, Jørgensen M. (2012). From origami to software development: A review of studies on judgment-based predictions of performance time. Psychol Bull 138(2), 238–271.
14. Love P, I LavagnonI. (2022). Making Sense of Hospital Project MisPerformance: Over Budget, Late,Time and Time Again—Why? And What Can Be Done About It? Engineering 5
15. House of Lords Economic Affairs Committee. (2009). Private Finance Projects and off-balance sheet debt - Economic Affairs Committee. https://publications.parliament.uk/pa/ld200910/ldselect/ldeconaf/63/09110309.htm?utm_source=chatgpt.com
16. Jørgensen M, Grimstad S. (2012). Software effort estimation: Maturity and improvement. Adv Computers, 86, 169–213.
17. Jørgensen M. (2004). Top-down and bottom-up expert estimation of software development effort. Inform Software Technol 46(1), 3–16.
18. Kruger J, Evans M. (2004). If you don't want to be late, enumerate: Unpacking reduces the planning fallacy. J Exp Psychol: Applied 10(3), 216–224.
19. Flyvbjerg B, Budzier A. (2011) Why your IT project may be riskier than you think. Harv Bus Rev 89(9):23–25.