Geodetic Foundations

The foundation of all navigation is understanding what we’re navigating ON. This module provides the rigorous mathematical framework used by the National Geodetic Survey, Defense Mapping Agency, and NATO geodesists.

Historical Context: How We Learned Earth’s Shape

Eratosthenes (~240 BCE)

First accurate measurement of Earth’s circumference using shadow angles:

\[C = \frac{360°}{\theta} \times d\]

Where: * \(\theta\) = angular difference in sun position = 7.2° * \(d\) = distance from Alexandria to Syene ≈ 800 km

\[C = \frac{360°}{7.2°} \times 800 = 40,000 \text{ km}\]

Remarkably accurate: Modern value = 40,075 km (equatorial)

Newton’s Prediction (1687)

Newton’s Principia predicted Earth is an oblate spheroid - flattened at poles due to rotation.

\[\frac{\text{Equatorial Radius}}{\text{Polar Radius}} > 1\]

French Geodetic Expeditions (1735-1744)

The Paris Academy sent expeditions to: * Lapland (Maupertuis) - measured 1° of latitude near Arctic * Peru (La Condamine) - measured 1° of latitude near Equator

Result: 1° of latitude is LONGER near the poles, confirming Newton.

Modern Era: Satellite Geodesy

  • 1957: Sputnik - first satellite tracking for geodesy

  • 1966: Naval Weapons Lab develops WGS 66

  • 1972: WGS 72

  • 1984: WGS 84 (current global standard)

  • 2013: WGS 84 (G1762) - latest realization

Earth’s True Shape

The Geoid

Earth is not a perfect sphere. It’s not even a perfect ellipsoid. The true shape is called the geoid - an equipotential surface of Earth’s gravity field that closely approximates mean sea level.

The geoid is lumpy and irregular because:

  • Mass distribution is uneven (mountains, ocean trenches, dense rock)

  • Gravity varies across Earth’s surface

  • The geoid can deviate ±100m from a reference ellipsoid
Geoid vs Ellipsoid
                    Geoid (irregular)
                   /
    _____....-----    -----...._____
   /   Ellipsoid (smooth mathematical)  \
  |                                      |
   \____________________________________/

Reference Ellipsoids

Since the geoid is mathematically complex, we approximate Earth with an ellipsoid (oblate spheroid) - a sphere flattened at the poles.

Ellipsoid Parameters
\[\text{Flattening: } f = \frac{a - b}{a}\]
\[\text{Eccentricity: } e = \sqrt{\frac{a^2 - b^2}{a^2}} = \sqrt{2f - f^2}\]

Where:

  • \(a\) = semi-major axis (equatorial radius)

  • \(b\) = semi-minor axis (polar radius)

Table 1. Historical Ellipsoids
Ellipsoid Semi-major (m) Semi-minor (m) Flattening Year

Clarke 1866

6,378,206.4

6,356,583.8

1/294.978

1866

Bessel 1841

6,377,397.155

6,356,078.963

1/299.153

1841

GRS 80

6,378,137.0

6,356,752.3141

1/298.257222

1980

WGS 84

6,378,137.0

6,356,752.3142

1/298.257223563

1984

WGS 84 (World Geodetic System 1984) is the global standard used by GPS.

Geodetic Datums

A datum is a reference frame for measuring locations on Earth. It consists of:

  1. A reference ellipsoid

  2. An origin point (where ellipsoid is anchored)

  3. An orientation (how it’s aligned with Earth’s rotation)

Why Datums Matter

Using the wrong datum can result in position errors of hundreds of meters.

Example: NAD 27 vs WGS 84 in Los Angeles = ~100m shift
Table 2. Common Datums
Datum Ellipsoid Usage

WGS 84

WGS 84

Global standard, GPS

NAD 83

GRS 80

North America (nearly identical to WGS 84)

NAD 27

Clarke 1866

Legacy North American maps

ED 50

International 1924

Europe (legacy)

OSGB 36

Airy 1830

Great Britain

Datum Transformations

Converting between datums requires transformation parameters:

7-Parameter Helmert Transformation
\[\begin{bmatrix} X' \\ Y' \\ Z' \end{bmatrix} = \begin{bmatrix} T_X \\ T_Y \\ T_Z \end{bmatrix} + (1 + s) \begin{bmatrix} 1 & -R_Z & R_Y \\ R_Z & 1 & -R_X \\ -R_Y & R_X & 1 \end{bmatrix} \begin{bmatrix} X \\ Y \\ Z \end{bmatrix}\]

Where:

  • \(T_X, T_Y, T_Z\) = translation (meters)

  • \(R_X, R_Y, R_Z\) = rotation (arcseconds)

  • \(s\) = scale factor (ppm)

Coordinate Systems

Geographic Coordinates (Geodetic)

Latitude (φ)

  • Angular distance from equator

  • Range: -90° (South Pole) to +90° (North Pole)

  • Positive = North, Negative = South

Longitude (λ)

  • Angular distance from Prime Meridian (Greenwich)

  • Range: -180° to +180° (or 0° to 360°)

  • Positive = East, Negative = West

Coordinate Notations

Format Example Notes

DD (Decimal Degrees)

33.7583, -117.8683

Best for computation

DMS (Degrees Minutes Seconds)

33°45'30"N, 117°52'06"W

Traditional, human-readable

DDM (Degrees Decimal Minutes)

33°45.500’N, 117°52.100’W

Maritime/aviation common

Conversion Formulas

DMS to Decimal Degrees
\[DD = D + \frac{M}{60} + \frac{S}{3600}\]
Decimal Degrees to DMS
\[D = \lfloor DD \rfloor\]
\[M = \lfloor (DD - D) \times 60 \rfloor\]
\[S = ((DD - D) \times 60 - M) \times 60\]
Example Conversion
Convert 33.7583°N to DMS:

D = floor(33.7583) = 33
M = floor((33.7583 - 33) × 60) = floor(45.498) = 45
S = (45.498 - 45) × 60 = 29.88 ≈ 30

Result: 33°45'30"N

Practice Problems

Problem 1: Convert 45°30'45"N, 122°40'30"W to decimal degrees.

Problem 2: Convert -33.8688, 151.2093 to DMS.

Problem 3: A map shows coordinates in NAD 27. Your GPS uses WGS 84. The transformation parameters are ΔX = -8m, ΔY = +160m, ΔZ = +176m. Which datum shows the point further north?

Geocentric vs Geodetic Coordinates

Geocentric (ECEF)

Earth-Centered, Earth-Fixed (ECEF) uses Cartesian coordinates (X, Y, Z) with origin at Earth’s center.

Geodetic to ECEF Conversion
\[N = \frac{a}{\sqrt{1 - e^2 \sin^2(\phi)}}\]
\[X = (N + h) \cos(\phi) \cos(\lambda)\]
\[Y = (N + h) \cos(\phi) \sin(\lambda)\]
\[Z = (N(1 - e^2) + h) \sin(\phi)\]

Where:

  • \(N\) = radius of curvature in the prime vertical

  • \(h\) = height above ellipsoid

  • \(e\) = ellipsoid eccentricity

When to Use Each

System Use Case Example

Geographic (φ, λ, h)

Human-readable, mapping

"Meet at 33.75°N, 117.87°W"

ECEF (X, Y, Z)

Satellite calculations, GPS internals

GPS receiver computations

Local Tangent Plane (ENU)

Local navigation, surveying

"Target is 500m East, 200m North"

Height Systems

"Height" is ambiguous. Always specify the reference surface.
Table 3. Types of Height
Height Type Definition

Ellipsoidal (h)

Height above reference ellipsoid (what GPS measures)

Orthometric (H)

Height above geoid (mean sea level) - what we intuit as "elevation"

Geoid Undulation (N)

Difference between geoid and ellipsoid: \(h = H + N\)
Height Relationship
\[h = H + N\]

Where:

  • \(h\) = ellipsoidal height (GPS)

  • \(H\) = orthometric height (elevation)

  • \(N\) = geoid undulation (lookup from model like EGM2008)

Example
GPS reports: h = 100m (ellipsoidal)
Geoid undulation at location: N = -30m
Actual elevation: H = h - N = 100 - (-30) = 130m above sea level

Geoid Height Models

The geoid is not static. It must be modeled mathematically using spherical harmonics.
Table 4. Geoid Models
Model Authority Resolution Notes

EGM84

NGA/DMA

180×180

Early GPS era

EGM96

NGA

360×360

10-20 cm accuracy, widely used

EGM2008

NGA

2159×2159

Current standard, ±10 cm globally

GEOID18

NGS

Variable

North America only, ±2 cm
Spherical Harmonic Representation
\[N(\phi, \lambda) = \sum_{n=0}^{N_{max}} \sum_{m=0}^{n} P_{nm}(\sin\phi)[C_{nm}\cos(m\lambda) + S_{nm}\sin(m\lambda)]\]

Where: * \(P_{nm}\) = fully normalized Legendre functions * \(C_{nm}, S_{nm}\) = spherical harmonic coefficients * \(N_{max}\) = maximum degree (2159 for EGM2008)

Table 5. Geoid Undulation Examples (WGS84/EGM2008)
Location Latitude Longitude N (meters)

Iceland (minimum)

65°N

18°W

-105.0

New Guinea (maximum)

5°S

145°E

+85.0

Los Angeles

34°N

118°W

-32.5

McAllen, TX

26°N

98°W

-26.0

Denver, CO

40°N

105°W

-15.5

Why Geoid Undulation Matters

A 30-meter error in elevation could:

  • Cause aircraft to think they’re 100 feet higher than reality

  • Make submarines calculate incorrect depth

  • Create flooding models that are dangerously wrong

  • Misalign precision-guided munitions

Radius of Curvature

Earth’s radius varies with latitude due to flattening.

Meridian Radius of Curvature (M)
\[M = \frac{a(1 - e^2)}{(1 - e^2 \sin^2(\phi))^{3/2}}\]
Prime Vertical Radius of Curvature (N)
\[N = \frac{a}{\sqrt{1 - e^2 \sin^2(\phi)}}\]
Mean Radius of Curvature ®
\[R = \sqrt{M \cdot N}\]
Table 6. WGS 84 Radii at Selected Latitudes
Latitude M (meters) N (meters) Mean R

0° (Equator)

6,335,439

6,378,137

6,356,752

45°

6,367,381

6,388,838

6,378,101

90° (Pole)

6,399,594

6,399,594

6,399,594

Vincenty Formula (Geodesic Distance on Ellipsoid)

The Haversine formula assumes a sphere. For military-grade accuracy, we use Vincenty’s formulae which account for the ellipsoid.

Vincenty achieves ±0.5 mm accuracy on the WGS84 ellipsoid - required for precision geodesy and guided munitions.

The Inverse Problem

Given two points \((\phi_1, \lambda_1)\) and \((\phi_2, \lambda_2)\), find distance \(s\) and azimuths \(\alpha_1, \alpha_2\).

Reduced Latitude
\[\tan U = (1 - f) \tan \phi\]
Difference in Longitude on Auxiliary Sphere
\[\lambda = L\]

Then iterate until convergence:

\[\sin \sigma = \sqrt{(\cos U_2 \sin \lambda)^2 + (\cos U_1 \sin U_2 - \sin U_1 \cos U_2 \cos \lambda)^2}\]
\[\cos \sigma = \sin U_1 \sin U_2 + \cos U_1 \cos U_2 \cos \lambda\]
\[\sigma = \arctan2(\sin \sigma, \cos \sigma)\]
\[\sin \alpha = \frac{\cos U_1 \cos U_2 \sin \lambda}{\sin \sigma}\]
\[\cos^2 \alpha = 1 - \sin^2 \alpha\]
\[\cos 2\sigma_m = \cos \sigma - \frac{2 \sin U_1 \sin U_2}{\cos^2 \alpha}\]
\[C = \frac{f}{16} \cos^2 \alpha [4 + f(4 - 3\cos^2 \alpha)]\]
\[\lambda' = L + (1 - C) f \sin \alpha \{\sigma + C \sin \sigma [\cos 2\sigma_m + C \cos \sigma (-1 + 2\cos^2 2\sigma_m)]\}\]

Iterate until \(|\lambda' - \lambda| < 10^{-12}\)

Final Distance Calculation
\[u^2 = \cos^2 \alpha \frac{a^2 - b^2}{b^2}\]
\[A = 1 + \frac{u^2}{16384}\{4096 + u^2[-768 + u^2(320 - 175u^2)]\}\]
\[B = \frac{u^2}{1024}\{256 + u^2[-128 + u^2(74 - 47u^2)]\}\]
\[\Delta\sigma = B \sin \sigma \{\cos 2\sigma_m + \frac{B}{4}[\cos \sigma(-1 + 2\cos^2 2\sigma_m) - \frac{B}{6}\cos 2\sigma_m(-3 + 4\sin^2 \sigma)(-3 + 4\cos^2 2\sigma_m)]\}\]
\[s = b \cdot A(\sigma - \Delta\sigma)\]

Forward Azimuths

\[\alpha_1 = \arctan2(\cos U_2 \sin \lambda, \cos U_1 \sin U_2 - \sin U_1 \cos U_2 \cos \lambda)\]
\[\alpha_2 = \arctan2(\cos U_1 \sin \lambda, -\sin U_1 \cos U_2 + \cos U_1 \sin U_2 \cos \lambda)\]

Comparison: Spherical vs Ellipsoidal

Table 7. LAX to JFK Distance Comparison
Method Distance Error vs Vincenty

Great Circle (sphere R=6371km)

3974.21 km

~3.8 km error

Haversine (sphere)

3974.21 km

~3.8 km error

Vincenty (WGS84)

3978.05 km

Reference (±0.5mm)

For distances under 100 km, Haversine is adequate. For artillery, aviation, and precision survey, always use Vincenty.

Exercises

Calculation Drill

  1. Calculate the eccentricity of the WGS 84 ellipsoid given:

    • a = 6,378,137 m

    • b = 6,356,752.3142 m

  2. Convert the following coordinates:

    • 40°26'46"N, 79°58'56"W → Decimal Degrees

    • -34.6037, -58.3816 → DMS

  3. A GPS reports ellipsoidal height of 250m. The geoid undulation at that location is +35m. What is the orthometric height (elevation)?

Conceptual Questions

  1. Why does the military use MGRS instead of latitude/longitude?

  2. If you have a map using NAD 27 datum and a GPS set to WGS 84, what practical problem arises?

  3. Why is the geoid not used directly as a reference surface for coordinates?

CLI Verification

# Install Python geospatial tools
pip install pyproj geopy

# Convert coordinates with Python
python3 << 'EOF'
from pyproj import CRS, Transformer

# WGS84 geographic to ECEF
wgs84 = CRS.from_epsg(4326)  # WGS84 geographic
ecef = CRS.from_epsg(4978)   # WGS84 ECEF

transformer = Transformer.from_crs(wgs84, ecef)
x, y, z = transformer.transform(33.7583, -117.8683, 100)
print(f"ECEF: X={x:.2f}, Y={y:.2f}, Z={z:.2f}")
EOF

Summary

Concept Key Point

Geoid

True shape of Earth, based on gravity; irregular

Ellipsoid

Mathematical approximation of Earth; smooth, defined by a, b, f

Datum

Reference frame = ellipsoid + origin + orientation

WGS 84

Global standard, used by GPS

Latitude

Angle from equator; -90° to +90°

Longitude

Angle from Prime Meridian; -180° to +180°

Ellipsoidal height

Height above ellipsoid (GPS raw)

Orthometric height

Height above geoid (elevation)

Next

Continue to Map Projections to learn how we flatten Earth onto paper.