Methods

How the Database of Battles is built: one ledger of cited claims, a crosswalk to Wikidata, and the Kursk Data Base read back out of a 1998 US Army report. For each step, the problems met and how each was solved.

427
engagements linked to Wikidata
105
Kursk units and formations
1,405
daily personnel returns
1,174
daily unit positions
1,350
daily weapon returns

Principles

1. Surveying comparable datasets

CDB90, the Army's database of 660 engagements from 1600 to 1982, was compared with its relatives: Dupuy's Land Warfare Database, the HERO reports in DTIC, the National Park Service Civil War battle summaries, Wikidata and DBpedia, Correlates of War, UCDP, ACLED, Brecke, Seshat and Bodart. Wikidata was the best general link target. The DTIC reports behind CDB90's successors held the richest primary data.

2. Wikidata crosswalk

A snapshot of about 11,700 Wikidata battles and sieges dated 1595–1975 was matched to CDB90 engagements. The match uses CDB90's DBpedia link, following Wikipedia redirects, or failing that the name and date.

ProblemMany engagements have no unambiguous match: a CDB90 engagement can be one phase of a Wikidata battle, or several battles at once.
SolutionOnly the 427 confident links were asserted. The other 233 went to a review page for people to resolve, with engagements that have battle pages first. Answers are imported as claims; reviewer disagreements are held back, not guessed.

3. Finding the Kursk sources

ProblemDTIC refused scripted downloads and was down for maintenance. The Kursk and Ardennes databases themselves (dBase files built by The Dupuy Institute) are not online.
SolutionThe Internet Archive mirrors the reports. KOSAVE II (CAA-SR-98-7, AD-A360-311, approved for public release) prints the Kursk Data Base's daily statistics in its appendices, so the database could be recovered from print. The PDF is pinned by checksum in a source manifest.

4. Daily personnel Appendices F–G

One table per unit: nine categories (on hand, total, combat and non-battle casualties, killed, missing, wounded, replaced, returned) by day, 4–18 July 1943, with row totals.

ProblemThe scans are skewed, so a printed row comes out split across text lines. Some pages come out column by column, and numbers break apart (1 5880).
SolutionCells are placed by position, not by line. Each word goes to the nearest row label vertically and to the day column its right edge sits under, after split numbers are rejoined.
ProblemOCR drops or misreads day-header numbers.
SolutionA least-squares fit through the header numbers that survived puts all 15 evenly spaced columns back.
ProblemRow labels come out scrambled or unreadable.
SolutionThe nine categories always print in the same order, so rows are assigned by position, and the even spacing fills in any label that can't be read.
ProblemThe text layer garbles five-digit strengths, and Vision skips lone zeros.
SolutionThe two readers cover each other. Vision overrides the text layer only when its value makes the row sum to its total or removes an implausible jump.

Checks: each row sums to its printed total; combat casualties = killed + missing + wounded; total = combat + non-battle; a day-to-day jump in on-hand strength of more than 2× is flagged for line units. A single unread cell is recovered from its row total. Result: 12,443 valid values; 701 of 703 rows match their totals. Two defects are in the print itself: the 11th and 19th Panzer Divisions' on-hand figures are truncated.

5. Order of battle

ProblemThe appendix expected to give the order of battle (Appendix D) lists only small attached units.
SolutionTables 3-3 and 3-4 give each unit's higher HQ on 4 July. Names were normalized across tables (332dID = 332nd ID).

The source puts the 167th Infantry Division under 48th Panzer Corps on 4 July, not II SS Panzer Corps as often given; the ledger follows the source. CDB90 engagements 493–497 are linked to the units on each side through this order of battle. Where CDB90 notes unnamed reinforcements ("6TH GDS ARMY(+)"), the claim says they are not enumerated.

6. Daily positions Appendix L

Each unit's daily location is the mean of its KDB reference points (HQ excluded), with the number of points behind it.

ProblemThe Soviet pages' text layer comes out column by column, and coordinates carry OCR noise (N 50 .563).
SolutionLines are rebuilt from word positions and read by Vision too. A noise-tolerant pattern parses each line, and all 1,174 printed entries are accounted for.
ProblemFuzzy name matching can confuse units: "52d RC" looks like "52d Gds RD".
SolutionThe unit number must match exactly first; only the rest of the name is compared fuzzily.

Checks: every position must fall inside the battle area, and a move of more than 40 km in a day needs both readers to agree. The one such move, the 2d Guards Tank Corps coming up from reserve on 5 July, is real. Five independent tank brigades appear only here, so they have positions but no personnel.

7. Daily weapons and tanks Appendices H–I

One table per unit, often over two or three pages, in six sections: on hand, destroyed or abandoned, damaged, in repair, replaced, returned. Each has a row per weapon type and a column per day.

ProblemWeapon rows have variable labels, some ending in digits (T-34, KV-1, SU-122).
SolutionValues are taken from the right, exactly 15 or 16 per row, and whatever remains is the label.
ProblemThe skew left each label 1–2 pt below its numbers, so rows split.
SolutionEach page's slope is estimated from its long lines and removed before grouping.
ProblemDigits run together (000010), one header prints its days as 1–15, and one TOTAL heading was lost.
SolutionRuns are split one digit per column by width; headers with 15 numbers are placed by order; a lost total column is restored at its usual offset.
ProblemGarbled weapon names: HOmmmtr, PzH, T-tfO.
SolutionRows print alphabetically, which places them: 120mm mortar, Panzer II and T-60. Every correction is in an explicit table.
ProblemOn-hand rows have no printed total to check.
SolutionWhere the readers disagree, the value without a 2× jump against its neighbours wins, which discards Vision's merged numbers (268268). A cell is dropped only when both readings are plausible.

Checks tried and kept only as reports: the day-to-day balance (on hand = yesterday − losses + replacements + returns) holds 93% of the time. The rest are mostly real one-day attachments, so it cannot reject values. Alphabetical order holds in 545 of 555 sections; the exceptions are printed that way. Result: 43,105 of 43,181 values valid. On 4 July the data gives about 1,000 German and 2,500 Soviet tanks on the southern face, in line with published figures.

8. Daily ammunition Appendices J–K

One table per unit, 92 in all: short tons on hand, used and received each day, for tank and antitank, artillery, rocket and other ammunition. Used and received print row totals; received goes negative when ammunition leaves with a detached unit.

ProblemTables break across pages without repeating the day header, and one page has no header at all.
SolutionThe unit carries over from the previous page, and a page with no header takes its columns from rows that print all sixteen values.
ProblemWithin one page, some tables space their last five days closer together than their own header.
SolutionA row with a value in every column is placed by order, so it does not need the grid; only incomplete rows are placed by position.
ProblemOn one page the labels sit 4–5 pt off their numbers, and zeros come out as ooooooooooooooo. Elsewhere 0-10 is two cells.
SolutionLabels and numbers left on separate lines are paired with the nearest one; runs of o are read as zeros, and a cell glued to a negative neighbour is split at the sign.
ProblemThe text layer drops lone zeros on 4–5 July.
SolutionUsed ammunition cannot be negative, so when the days read already reach the row total, the gaps must be zero.
ProblemVision reads a column of numbers down several rows as one line.
SolutionOnly Vision boxes one line tall are used. Here the text layer carries the read, and Vision confirms about 1,100 cells.

Checks: used and received rows must sum to their totals. Where the readers disagree on an on-hand cell, the value that keeps the day's balance (yesterday's on hand − used + received) wins. That balance holds exactly on 79% of days and to within a ton on 90%: the rest is rounding and the stocks of attached units, mostly at headquarters. Result: 16,425 of 16,560 printed values valid; the rest are rows whose own arithmetic cannot settle them.

9. The Kursk map

An export built from the resolved ledger feeds the Kursk page. It shows each unit's daily position, with circle area for men, for tanks plus self-propelled guns, or for tons of ammunition on hand. A day slider, unit tracks and an engagement filter go with it, and every value links to its scanned page.

ProblemDeciding headquarters from names made tank corps into formations.
SolutionA unit is a formation only if the order of battle gives it subordinates; tank and mechanized corps are KDB primary units.
ProblemThe full weapon breakdown is too large to load with the map.
SolutionThe map file carries daily armour totals; the full table loads only when a unit is opened.

10. The Ardennes: German daily casualties ACSDB report

The Ardennes Campaign Simulation Data Base (1990) is a dBASE database. Its report, the only part that is public, describes how the data were built and reprints a few tables. Two have daily detail: killed, wounded and captured for the 42 German divisions and brigades, 16 December 1944 to 16 January 1945, and each one's daily activity rating. There are no daily locations or equipment, so there is no map; the Ardennes page charts the casualties by unit and day.

These are estimates, not wartime returns. The report took 10-day and monthly army and division totals, divided them among divisions and spread them over days, an active day counting 6.5 times an inactive one. That is why a division can show 38 killed, 115 wounded and 34 captured ten days running. Where the report marks a day as resting on daily primary-source figures, the claim says so and carries higher confidence; the rest are cited as ACSDB estimates.

ProblemA dot-matrix printout: OCR turns digits into letters (1IS for 115, is for 15). Vision skips half the cells on a full page.
SolutionLetters are read back as digits, but a value that needed that is trusted only if Vision agrees or it continues a run of equal days. Vision reads a crop of the value columns, which it reads far more completely.
ProblemScans tilt by up to a row across the page and sit 30 pt apart from one page to the next.
SolutionThe tilt is measured from each record number to its date, then to the rightmost values; all positions are taken relative to the date column.
ProblemUnits join late or leave early, a page can lose a row, and both the running record numbers and the dates are garbled in plausible ways (5256 for 5252, 01/10/45 for 01/05/45).
SolutionRecord numbers, the text layer's date and Vision's date each vote for a row's day, and a small dynamic program picks the increasing sequence that agrees with the most votes. The activity tables are placed the same way from their day labels.
ProblemPages are not in the order the units are listed.
SolutionEach page's unit is read from its rows by majority.

Checks: the two tables check each other. Out-of-contact days average 2 casualties (155 of 169 are zero), inactive days 29, active days 157; units rated outside the area have no casualties (three edge days print zeros). A change in one column alone, read by one reader only, is dropped, since a real switch moves all three together. Result: 3,585 of 3,741 values for 1,238 unit-days, and 1,238 activity ratings.

11. Civil War battles CWSAC summaries

The Civil War Sites Advisory Commission (1993) named 384 principal battles of the American Civil War and wrote a one-page summary of each: other names, county, campaign, dates, principal commanders, forces engaged, estimated casualties, result, and a preservation priority with a significance class from A (decisive) to D (local). The National Park Service published the summaries until about 2020; they are gone from nps.gov, so each page was read from the Wayback Machine's latest good capture. The capture time, size and sha256 of every page are recorded, so the same bytes can be fetched again. The Civil War page charts them by date and casualties.

ProblemFigures are written, not tabulated: a total with each side in parentheses, a count of men inside a unit description, killed, wounded and missing written together, or a range.
SolutionEach side's number is read where the page gives one, killed, wounded and missing are summed, and a range is left unknown. A total is summed from the sides only when every side named is known.
ProblemThe pages have their own errors. LA003 prints reference AL001 and gives LA003 as its result; WV016 is titled with another battle's name (Cane Hill, Arkansas) and prints its priority as its result; SC008 prints a second priority in its reference field; three give casualties by side that do not add up to their total.
SolutionThe page's address is the battle's identity, and its state is the reference's prefix. Misplaced values are not used. WV016's other names, county, dates and reference are all Shepherdstown's, so it is named Shepherdstown. Every such error is noted on the claim it affects.
ProblemVA020 is one page for two battles, Glendale and White Oak Swamp; GA013A is an archived copy of GA013.
SolutionVA020 stays one battle with a priority for each; the copy is dropped. 382 pages cover the 384 battles.

Checks: every value was compared with an independent cleaning of the same pages (Jeffrey Arnold's acw_battle_data): dates differ for 1 battle, casualty totals for 2, results for 1 and class for 1, each where the archived page says what is used here. CDB90's 49 Civil War engagements are each linked to the CWSAC battle whose dates overlap and whose name is closest; all 49 agree with Arnold's crosswalk. Result: 382 battles, 340 with a casualty figure, 84 with forces as numbers.

The summaries' text is not published. The site shows the values parsed from it. The wording of every section, the battle descriptions included, is kept internally for classification, and any summary shown here will be written by us after analysis.

12. Civil War battlefields ABPP boundaries

The National Park Service's American Battlefield Protection Program maps each CWSAC battlefield as two areas: the core area, where the fighting was, and the study area, the battle's historic setting. The 1993 survey drew them first and the state resurveys of 2007–2013 redrew most of them. Both are in one public NPS map service, keyed by the CWSAC reference. The four Civil War layers were downloaded with their sizes and sha256 recorded. The layers carry names, states and acreage only, no text.

ProblemThe full-detail boundaries are 17 MB, too large for the page or the ledger.
SolutionEach area is simplified to about 50 metres, with coordinates to about a metre. The acreage is the layer's own, from the full-detail shape.
ProblemSome sites are in one survey and not the other, and the 1993 survey splits some battles the resurvey maps whole (the Marietta operations, Glendale).
SolutionThe resurvey is used wherever it has the site, and the 1993 survey only where it does not (6 sites). A site's two areas always come from the same survey. Sites with no CWSAC summary page (Bayou Meto, Ebenezer Church and the 1993 split sites) are left out. Bayou Fourche is AR010 in the layers and AR010A in the summaries.

Checks: every site's state matches its battle's. The point drawn for each battle lies inside its core area, or inside its study area where no core area is mapped. Result: 380 of the 382 battles are mapped. Round Mountain and Middle Boggy Depot, both in Oklahoma, have no boundary.

13. Revolutionary War and War of 1812 ABPP 2007 report

The National Park Service's 2007 report to Congress on Revolutionary War and War of 1812 sites rated every principal battlefield of both wars. Its preservation priority tables give each battlefield's site code, name, states, significance class (A to D), short- and long-term threat and priority group. They were read from the PDF, whose sha256 is recorded, and the 2007 study's boundaries come from the same NPS map service as the Civil War's. The Revolution and 1812 page maps them.

ProblemThe PDF's text splits letters ("T renton", "V A207") and wraps long names onto a second line.
SolutionThe splits are rejoined before parsing, and a line that continues the row above it is added to that row's name. All 243 battlefields parse: 165 of the Revolution and 78 of 1812, the report's own totals.
ProblemThe report gives no dates.
SolutionEach battlefield is matched to a Wikidata battle of its war. Both must be each other's best match, the names must agree closely, and the battle's coordinates must fall within or near the battlefield's boundary. Where several battlefields share a name (three Charlestons, several Savannahs), only the qualifier in brackets counts. A date Wikidata gives only to the year is kept as the year. 106 battlefields are dated; the rest are left undated rather than guessed.
ProblemA few boundary records are faulty: Lake Erie's core area is a speck near the equator, and two Virginia core areas have negative acreage.
SolutionShapes with no real area are dropped, and negative acreages are treated as not given.

Result: 243 battlefields, 236 mapped and 106 dated. Only the battlefields are read, not the forts, depots and other associated properties the report also rates, nor the sites in Canada.

14. Civil War orders of battle Official Records

The Official Records of the Union and Confederate Armies print, for each major battle, the organization of both armies: corps, divisions and brigades, each with its commanders in order of succession, then its regiments and batteries with theirs. The orders of battle are read from the Internet Archive's scans, whose OCR gives a box for every word; the files' sha256 are recorded. Gettysburg is first, from series I, volume 27, parts 1 and 2. The order of battle page shows both armies.

ProblemThe lists are set in two columns broken by centred headings, and the OCR reads straight across the page.
SolutionEach line is placed by its words' positions: a line that crosses the centre of the page is a heading, and between headings the left column is read before the right. Footnotes are set apart from the first footnote mark in the lower half of the page.
ProblemThe OCR misreads marks and numbers: "1lth" for 11th, "Bd" for 3d, a footnote dagger as a letter, a commander's name split over two lines.
SolutionThese are corrected by rule, and wrapped lines are rejoined. A name too garbled to read is cut at the first character no name contains.

Checks: the counts match the armies' known strength (Army of the Potomac: 7 infantry corps, 22 divisions, 250 infantry units and 74 batteries; Army of Northern Virginia: 3 corps, 10 divisions, 175 infantry units and 70 batteries). Against the National Park Service's list of Union regiments at Gettysburg, 259 of its 266 are found. Of the 7 it has that are not found here, two were not at Gettysburg, and the rest are OCR errors in a number (a "22d" for the 122d New York). Result: 844 formations and units on both sides.

The volumes' text is not published. Every page of each volume read so far (reports, correspondence, returns and orders of battle) is kept internally as text in reading order, with its footnotes and its numbered document, for classification and for the summaries we will write. The site shows only the values parsed from it.

Still open

Sources

ReportWhat it givesCopy
CAA-SR-98-7, KOSAVE II (1998), AD-A360-311Kursk Data Base: personnel, weapons, ammunition, locations, order of battle Internet Archive
The Ardennes Campaign Simulation Data Base (1990), AD-A240-088German daily battle casualties (estimates) and activity ratings Internet Archive
Civil War Sites Advisory Commission, battle summaries (1993)384 principal Civil War battles: dates, commanders, forces, casualties, result, class Wayback Machine
American Battlefield Protection Program, battlefield boundaries (1993; state resurveys 2007–2013)Core and study areas of 380 Civil War battlefields NPS map service
National Park Service, report to Congress on Revolutionary War and War of 1812 sites (2007)243 battlefields: significance class, threats, preservation priority NPS DataStore
CDB90660 engagements, 1600–1982US Army Concepts Analysis Agency
WikidataBattle identifiers and links wikidata.org