Preface

Why this book exists

I wrote this book because I kept seeing the same gap on drilling and completions teams: the field generates enormous amounts of written operational knowledge — Daily Drilling Reports, end-of-well reports, morning reports, NPT logs — and almost none of it is usable by the next engineer who needs it (Dawson and Verkuil 2014). It sits in inconsistently named PDFs — scanned or digital — scattered across SharePoint folders and email threads. When the assembly loses tool face mid-slide and gets stuck downhole, the fastest way to find out if this has happened before is often to ask the engineer who’s been on the project the longest, hoping they remember a similar incident.

This book’s own archive has exactly that event — a real stuck-pipe day, verified against the original report. But dropping the archive’s PDFs into a folder and searching them doesn’t solve it either: searching "stuck pipe" finds the report where the pipe actually got stuck — but misses the very next day’s report entirely, because that one talks about "tight hole," "high torque," and a "decision to pull out of hole" instead. Same event, continuing, one day later — invisible to anything that only matches on exact words. That gap, between what a report says and what it means, is exactly what retrieval-augmented generation (RAG) closes.

Large language models are genuinely useful here — not because they are new or exciting, but because they solve a real, boring, expensive problem: turning an unstructured archive into something you can query. This book teaches you to build that system yourself, understand every part of it, and trust its output, rather than adopting a black-box product you cannot inspect.

What this protects

The gap this book closes isn’t really about PDFs. It’s about what happens when the person who remembers a similar incident retires, changes companies, or is simply on a different rig this week. A DDR archive is, in a sense, tribal knowledge that hasn’t been asked to prove itself yet — findable only if you already know which report to open, or who to call.

A system like the one you’ll build here doesn’t replace that engineer’s judgment. It replaces the search. Deciding what a stuck-pipe signature means, or whether a packer failure changes tomorrow’s plan, is still yours to make — the system’s job is only to put the right evidence in front of you fast enough that the decision doesn’t wait on someone’s memory, or on however long it takes to open dozens of PDFs one at a time. That matters most exactly where it’s easiest to overlook: getting an engineer new to a well up to speed on events they weren’t there for, and investigations where the fastest path to an answer today is still “ask the person who’s been here longest” — which stops working the day that person isn’t.

Why “industrial”

A demo that answers one question about one PDF is easy to build and easy to find online. An industrial system is different. It has to:

Handle scanned reports as well as digital ones.
Scale to thousands of DDRs across dozens of wells.
Never silently invent an answer — every claim must trace back to a specific report and page.
Be auditable by someone who was not in the room when it was built.

Every chapter in Part II exists because a toy RAG system fails at one of these requirements. We fix them one at a time, in the order you would actually hit them in the field.

How to read this book

This book borrows the practical-first, project-based spirit of Python Crash Course (Matthes 2023) — small wins early, theory only when the code in front of you needs it, a working artifact at the end of every chapter — but assumes considerably less going in. You do not need to have written Python before. You do not need to know what a function or a loop is before Chapter 1; each concept is explained in plain language, translated into terms an engineer already knows, exactly when you need it to solve the problem in front of you.

If you’ve never installed Python or set up a project folder before, start with Part 0 — it gets you to a working setup in whichever editor or notebook tool you’re most comfortable with, with no assumption that you’ve done this before.

Read it in order the first time. Each chapter assumes the code from the previous one exists and works — this is a build, not a reference manual. After your first pass, use it as a reference: jump to the chapter that matches the problem you currently have (Chapter 6 if your PDFs are scanned, Chapter 9 if your retrieval quality is poor, Chapter 10 if your answers aren’t trustworthy yet).

Type the code yourself. Do not copy-paste from the companion repository until you have typed it once and watched it run. This is slower and it is worth it — and if you finish this book able to say “I can build this,” rather than “I understand the theory,” it did its job.

A note on data

Most Daily Drilling Reports are operationally sensitive and confidential — which is exactly why this book doesn’t use one from a commercial operator. Every example here comes from Utah FORGE, a Department of Energy-funded enhanced geothermal system research well (FORGE 16A(78)-32) whose reports are public: real well name, real dates, real personnel, real events, exactly as filed. No anonymisation was applied, because none was needed — this is what “publicly available” means in practice, and it lets this book show you genuine field language instead of an approximation of it.

That said, the technique generalises directly to a confidential archive: everything from Chapter 6 onward — OCR gating, traceability, corpus-completeness checks — exists precisely because production archives are messier and more sensitive than this one. If you point this book’s code at your own organisation’s DDRs, do not commit them to a public repository, and see Appendix A for a data-handling checklist.

# Preface {.unnumbered} ## Why this book exists I wrote this book because I kept seeing the same gap on drilling and completions teams: the field generates enormous amounts of written operational knowledge — Daily Drilling Reports, end-of-well reports, morning reports, NPT logs — and almost none of it is *usable* by the next engineer who needs it [@dawson2014ddr]. It sits in inconsistently named PDFs — scanned or digital — scattered across SharePoint folders and email threads. When the assembly loses tool face mid-slide and gets stuck downhole, the fastest way to find out if this has happened before is often to ask the engineer who's been on the project the longest, hoping they remember a similar incident. This book's own archive has exactly that event — a real stuck-pipe day, verified against the original report. But dropping the archive's PDFs into a folder and searching them doesn't solve it either: searching `"stuck pipe"` finds the report where the pipe actually got stuck — but misses the very next day's report entirely, because that one talks about `"tight hole,"` `"high torque,"` and a `"decision to pull out of hole"` instead. Same event, continuing, one day later — invisible to anything that only matches on exact words. That gap, between what a report *says* and what it *means*, is exactly what retrieval-augmented generation (RAG) closes. Large language models are genuinely useful here — not because they are new or exciting, but because they solve a real, boring, expensive problem: turning an unstructured archive into something you can query. This book teaches you to build that system yourself, understand every part of it, and trust its output, rather than adopting a black-box product you cannot inspect. ## What this protects The gap this book closes isn't really about PDFs. It's about what happens when the person who remembers a similar incident retires, changes companies, or is simply on a different rig this week. A DDR archive is, in a sense, tribal knowledge that hasn't been asked to prove itself yet — findable only if you already know which report to open, or who to call. A system like the one you'll build here doesn't replace that engineer's judgment. It replaces the search. Deciding what a stuck-pipe signature means, or whether a packer failure changes tomorrow's plan, is still yours to make — the system's job is only to put the right evidence in front of you fast enough that the decision doesn't wait on someone's memory, or on however long it takes to open dozens of PDFs one at a time. That matters most exactly where it's easiest to overlook: getting an engineer new to a well up to speed on events they weren't there for, and investigations where the fastest path to an answer today is still "ask the person who's been here longest" — which stops working the day that person isn't. ## Why "industrial" A demo that answers one question about one PDF is easy to build and easy to find online. An industrial system is different. It has to: - Handle scanned reports as well as digital ones. - Scale to thousands of DDRs across dozens of wells. - Never silently invent an answer — every claim must trace back to a specific report and page. - Be auditable by someone who was not in the room when it was built. Every chapter in Part II exists because a toy RAG system fails at one of these requirements. We fix them one at a time, in the order you would actually hit them in the field. ## How to read this book This book borrows the practical-first, project-based spirit of *Python Crash Course* [@matthes2023python] — small wins early, theory only when the code in front of you needs it, a working artifact at the end of every chapter — but assumes considerably less going in. You do not need to have written Python before. You do not need to know what a function or a loop is before Chapter 1; each concept is explained in plain language, translated into terms an engineer already knows, exactly when you need it to solve the problem in front of you. If you've never installed Python or set up a project folder before, start with Part 0 — it gets you to a working setup in whichever editor or notebook tool you're most comfortable with, with no assumption that you've done this before. Read it in order the first time. Each chapter assumes the code from the previous one exists and works — this is a build, not a reference manual. After your first pass, use it as a reference: jump to the chapter that matches the problem you currently have (Chapter 6 if your PDFs are scanned, Chapter 9 if your retrieval quality is poor, Chapter 10 if your answers aren't trustworthy yet). Type the code yourself. Do not copy-paste from the companion repository until you have typed it once and watched it run. This is slower and it is worth it — and if you finish this book able to say "I can build this," rather than "I understand the theory," it did its job. ## A note on data Most Daily Drilling Reports are operationally sensitive and confidential — which is exactly why this book doesn't use one from a commercial operator. Every example here comes from **Utah FORGE**, a Department of Energy-funded enhanced geothermal system research well (FORGE 16A(78)-32) whose reports are public: real well name, real dates, real personnel, real events, exactly as filed. No anonymisation was applied, because none was needed — this is what "publicly available" means in practice, and it lets this book show you genuine field language instead of an approximation of it. That said, the *technique* generalises directly to a confidential archive: everything from Chapter 6 onward — OCR gating, traceability, corpus-completeness checks — exists precisely because production archives are messier and more sensitive than this one. If you point this book's code at your own organisation's DDRs, do not commit them to a public repository, and see [Appendix A](appendix/appendix_a_environment_setup.qmd) for a data-handling checklist.