Appendix A: Environment Setup

Part 0 covers everything you need to install Python, create a virtual environment, and run the book’s chapter scripts — start there if you haven’t already. This appendix covers a few things Part 0 doesn’t: the sample dataset’s layout, rendering this book itself (not just running its code), the companion DDR Intelligence Platform pipeline referenced in Chapters 6–12, a data-handling note for your own archives, and troubleshooting for a few less common errors.

If you’d like a guide specific to the editor or notebook environment you chose in Part 0, see Appendices A1–A5 instead.

1. The sample dataset

Chapters 1–5 use ten real, curated Daily Drilling Reports from the public Utah FORGE archive (well FORGE 16A(78)-32) — already committed as PDFs in datasets/sample_ddrs/, since this is public data with no confidentiality concerns. Part II’s “at scale” chapters use the full 76-report archive, committed at datasets/forge_archive/. Nothing needs to be generated or downloaded to start: cloning the repository (Part 0, Section 0.6) gives you the full dataset.

If you want to reproduce either curated subset from your own copy of the full public archive, or extend it with more reports, see code/chapter_01/build_sample_archive.py and code/chapter_08/build_full_archive.py.

2. Rendering the book with Quarto

Running the chapter scripts (Part 0) doesn’t require anything beyond Python. Rebuilding this book itself — the HTML and PDF you’re reading — is a separate step that needs Quarto 1.7 or later:

quarto --version

If that’s not recognised, install Quarto from quarto.org for your operating system.

With Quarto installed and your virtual environment active:

quarto render

Output is written to _book/. For live-editing a single chapter while you write:

quarto preview chapters/chapter_01.qmd

Or, if you prefer conda/mamba instead of the .venv approach from Part 0:

conda env create -f environment.yml
conda activate ddr-rag-book

3. The companion pipeline (for Part II)

Chapters 6–12 reference DDR_UTAH_FORGE, a real, working DDR intelligence pipeline built specifically against this book’s public archive — same architecture as the reference platform this project grew from (per-document extraction, hybrid retrieval, structured facts, causality analysis), adapted for Utah FORGE’s real filenames and data. It’s a separate repository — clone it alongside this book’s repository, not inside it:

cd ../..                   # up and out of this book's own repository
git clone https://github.com/djimrastephane/DDR_UTAH_FORGE.git
cd DDR_UTAH_FORGE
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

Its own data/ folder is gitignored — the repository is code only. Copy this book’s datasets/forge_archive/ into DDR_UTAH_FORGE/data/raw/ (or point it at your own copy of the public Utah FORGE archive) and follow its README.md Quickstart to run the full pipeline and Streamlit dashboard.

You don’t need it to follow along

Part II chapters are written so every technique works fully standalone: the simplified code in code/chapter_06/ through code/chapter_12/ in this book’s repository requires no access to the companion pipeline. Where a chapter cites a specific file or a specific result from DDR_UTAH_FORGE (for example, the real 1,428-chunk global index in Chapter 8, or the real report-gap dates in Chapter 10), that result was checked against the pipeline’s actual output before being written into the book, so you can trust the numbers even without running the pipeline yourself.

4. A note on data handling

This book’s own data (datasets/sample_ddrs/, datasets/forge_archive/) is public and safe to commit, share, and publish — that’s true throughout this repository. The moment you point this book’s code at your own organisation’s DDRs instead, that changes:

Never commit real, confidential DDR PDFs, extracted text, or derived artefacts (embeddings, indexes) to a public repository.
If you’re experimenting on a shared machine, confirm your organisation’s data classification policy covers running third-party Python packages (embedding models, OCR engines) against confidential well data.
Add your own archive’s path to .gitignore before dropping any confidential PDFs into a local datasets/ folder for testing.

5. Troubleshooting

Part 0, Section 0.11 covers the common setup errors (Python not found, virtual environment not active, and so on). A few more specific to rendering the book or running Part II’s heavier packages:

Symptom	Likely cause
`quarto render` fails on a code cell	Activate `.venv` before rendering — Quarto uses whichever Python is on `PATH`. Confirm with `which python` and `python -c "import pdfplumber"`.
`ModuleNotFoundError: sentence_transformers` (Chapter 4+)	Re-run `pip install -r requirements.txt`; this is a heavier dependency and sometimes needs a separate `pip install torch` first on some platforms.
FAISS import error on Apple Silicon (Chapter 8+)	Ensure you installed `faiss-cpu`, not `faiss-gpu` — the latter has no macOS wheel.
`read_ddr.py` reports no text extracted	Confirm you’re pointing at a file in `datasets/sample_ddrs/` or `datasets/forge_archive/` — both are committed directly, so this shouldn’t happen unless the path is wrong.

# Appendix A: Environment Setup {#sec-appendix-a .unnumbered} [Part 0](../chapters/chapter_00.qmd) covers everything you need to install Python, create a virtual environment, and run the book's chapter scripts — start there if you haven't already. This appendix covers a few things Part 0 doesn't: the sample dataset's layout, rendering this book itself (not just running its code), the companion DDR Intelligence Platform pipeline referenced in Chapters 6–12, a data-handling note for your own archives, and troubleshooting for a few less common errors. If you'd like a guide specific to the editor or notebook environment you chose in Part 0, see Appendices A1–A5 instead. ## 1. The sample dataset Chapters 1–5 use ten real, curated Daily Drilling Reports from the public Utah FORGE archive (well FORGE 16A(78)-32) — already committed as PDFs in `datasets/sample_ddrs/`, since this is public data with no confidentiality concerns. Part II's "at scale" chapters use the full 76-report archive, committed at `datasets/forge_archive/`. Nothing needs to be generated or downloaded to start: cloning the repository (Part 0, Section 0.6) gives you the full dataset. If you want to reproduce either curated subset from your own copy of the full public archive, or extend it with more reports, see `code/chapter_01/build_sample_archive.py` and `code/chapter_08/build_full_archive.py`. ## 2. Rendering the book with Quarto Running the chapter scripts (Part 0) doesn't require anything beyond Python. Rebuilding this book itself — the HTML and PDF you're reading — is a separate step that needs [Quarto](https://quarto.org) 1.7 or later: ```bash quarto --version ``` If that's not recognised, install Quarto from [quarto.org](https://quarto.org) for your operating system. With Quarto installed and your virtual environment active: ```bash quarto render ``` Output is written to `_book/`. For live-editing a single chapter while you write: ```bash quarto preview chapters/chapter_01.qmd ``` Or, if you prefer conda/mamba instead of the `.venv` approach from Part 0: ```bash conda env create -f environment.yml conda activate ddr-rag-book ``` ## 3. The companion pipeline (for Part II) Chapters 6–12 reference [**DDR_UTAH_FORGE**](https://github.com/djimrastephane/DDR_UTAH_FORGE), a real, working DDR intelligence pipeline built specifically against this book's public archive — same architecture as the reference platform this project grew from (per-document extraction, hybrid retrieval, structured facts, causality analysis), adapted for Utah FORGE's real filenames and data. It's a separate repository — clone it alongside this book's repository, not inside it: ```bash cd ../.. # up and out of this book's own repository git clone https://github.com/djimrastephane/DDR_UTAH_FORGE.git cd DDR_UTAH_FORGE python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt ``` Its own `data/` folder is gitignored — the repository is code only. Copy this book's `datasets/forge_archive/` into `DDR_UTAH_FORGE/data/raw/` (or point it at your own copy of the public Utah FORGE archive) and follow its `README.md` Quickstart to run the full pipeline and Streamlit dashboard. ::: {.callout-tip title="You don't need it to follow along"} Part II chapters are written so every technique works fully standalone: the simplified code in `code/chapter_06/` through `code/chapter_12/` in *this* book's repository requires no access to the companion pipeline. Where a chapter cites a specific file or a specific result from `DDR_UTAH_FORGE` (for example, the real 1,428-chunk global index in Chapter 8, or the real report-gap dates in Chapter 10), that result was checked against the pipeline's actual output before being written into the book, so you can trust the numbers even without running the pipeline yourself. ::: ## 4. A note on data handling This book's own data (`datasets/sample_ddrs/`, `datasets/forge_archive/`) is public and safe to commit, share, and publish — that's true throughout this repository. The moment you point this book's code at your own organisation's DDRs instead, that changes: - Never commit real, confidential DDR PDFs, extracted text, or derived artefacts (embeddings, indexes) to a public repository. - If you're experimenting on a shared machine, confirm your organisation's data classification policy covers running third-party Python packages (embedding models, OCR engines) against confidential well data. - Add your own archive's path to `.gitignore` before dropping any confidential PDFs into a local `datasets/` folder for testing. ## 5. Troubleshooting Part 0, Section 0.11 covers the common setup errors (Python not found, virtual environment not active, and so on). A few more specific to rendering the book or running Part II's heavier packages: | Symptom | Likely cause | |---|---| | `quarto render` fails on a code cell | Activate `.venv` before rendering — Quarto uses whichever Python is on `PATH`. Confirm with `which python` and `python -c "import pdfplumber"`. | | `ModuleNotFoundError: sentence_transformers` (Chapter 4+) | Re-run `pip install -r requirements.txt`; this is a heavier dependency and sometimes needs a separate `pip install torch` first on some platforms. | | FAISS import error on Apple Silicon (Chapter 8+) | Ensure you installed `faiss-cpu`, not `faiss-gpu` — the latter has no macOS wheel. | | `read_ddr.py` reports no text extracted | Confirm you're pointing at a file in `datasets/sample_ddrs/` or `datasets/forge_archive/` — both are committed directly, so this shouldn't happen unless the path is wrong. |