Navigation Mathematics

The mathematical foundations of navigation span from basic trigonometry to advanced spherical geometry. This module provides the rigorous treatment expected at the Naval Postgraduate School and Army Command and General Staff College.

Prerequisite: Angle Measurement

Degrees, Minutes, Seconds

\[1° = 60' \text{ (minutes)}\]
\[1' = 60'' \text{ (seconds)}\]
\[1° = 3600''\]

Radians

\[\pi \text{ radians} = 180°\]
\[1 \text{ radian} = \frac{180°}{\pi} \approx 57.2958°\]
Conversion
\[\text{radians} = \text{degrees} \times \frac{\pi}{180}\]
\[\text{degrees} = \text{radians} \times \frac{180}{\pi}\]

Military Angular Measurement: Mils

Military forces worldwide use mils (milliradians) for artillery, mortar, and precision fire calculations. The mil system provides finer angular resolution than degrees for fire direction.

Definition

A mil (NATO standard) divides a circle into 6400 mils:

\[1 \text{ circle} = 6400 \text{ mils} = 360° = 2\pi \text{ rad}\]
\[1 \text{ mil} = \frac{360°}{6400} = 0.05625°\]
\[1° = \frac{6400}{360} \approx 17.778 \text{ mils}\]

Different Mil Standards:

Standard Mils/Circle Used By

NATO mil

6400

NATO forces, US military

Soviet/Russian mil

6000

Russia, former Warsaw Pact

Swedish streck

6300

Sweden, Finland

True milliradian

2000π ≈ 6283

Mathematical definition

The Mil Relation Formula

The fundamental relationship for range estimation and fire adjustment:

\[W = R \times \frac{m}{1000}\]

Where:

  • \(W\) = width of target (meters)

  • \(R\) = range to target (meters)

  • \(m\) = angular measurement (mils)

Rearranged for range estimation:

\[R = \frac{W \times 1000}{m}\]
Example: Range Estimation

A tank is observed. Known width = 3.5 meters. Measured angle = 2 mils.

\[R = \frac{3.5 \times 1000}{2} = 1750 \text{ meters}\]

Fire Adjustment

For lateral adjustments in artillery:

\[\text{Deflection (mils)} = \frac{\text{Lateral Error (m)} \times 1000}{\text{Range (m)}}\]
Example: Adjusting Fire

Round impacts 50m left of target at 2000m range.

\[\text{Correction} = \frac{50 \times 1000}{2000} = 25 \text{ mils RIGHT}\]

Conversion Factors

Conversion Formula

Degrees → NATO mils

\(\text{mils} = \text{degrees} \times 17.778\)

NATO mils → degrees

\(\text{degrees} = \text{mils} \times 0.05625\)

Radians → NATO mils

\(\text{mils} = \text{radians} \times 1018.59\)

NATO mils → radians

\(\text{radians} = \text{mils} \times 0.0009817\)

Planar Trigonometry Review

Right Triangle

        C
       /|
      / |
   c /  | a
    /   |
   /    |
  /_____|
 A   b   B

Where:
- A, B, C are angles
- a, b, c are opposite sides
- C = 90° (right angle)

SOHCAHTOA

\[\sin(\theta) = \frac{\text{opposite}}{\text{hypotenuse}} = \frac{a}{c}\]
\[\cos(\theta) = \frac{\text{adjacent}}{\text{hypotenuse}} = \frac{b}{c}\]
\[\tan(\theta) = \frac{\text{opposite}}{\text{adjacent}} = \frac{a}{b}\]

Pythagorean Theorem

\[c^2 = a^2 + b^2\]

Law of Sines

For any triangle:

\[\frac{a}{\sin(A)} = \frac{b}{\sin(B)} = \frac{c}{\sin(C)}\]

Law of Cosines

\[c^2 = a^2 + b^2 - 2ab\cos(C)\]

Spherical Trigonometry

Earth is a sphere (approximately). Spherical trigonometry deals with triangles on the surface of a sphere.

Spherical Triangle

        North Pole
           /\
          /  \
         /    \
        /      \
    A  /________\ B
      \          /
       \   C    /
        \      /
         \    /
          \  /
           \/
         Equator
  • Vertices are points on the sphere

  • Sides are arcs of great circles

  • Side lengths are measured as angles (central angles)

Spherical Law of Cosines (Sides)

\[\cos(c) = \cos(a)\cos(b) + \sin(a)\sin(b)\cos(C)\]

Spherical Law of Cosines (Angles)

\[\cos(C) = -\cos(A)\cos(B) + \sin(A)\sin(B)\cos(c)\]

Spherical Law of Sines

\[\frac{\sin(A)}{\sin(a)} = \frac{\sin(B)}{\sin(b)} = \frac{\sin(C)}{\sin(c)}\]

Napier’s Rules for Right Spherical Triangles

For a spherical triangle with one right angle (C = 90°), Napier’s Rules provide elegant mnemonic formulas. Named after John Napier (1550-1617), inventor of logarithms, these rules are fundamental to celestial navigation.

The Napier Circle

Arrange the five parts (excluding the right angle) in a circle:

            co-c
          /      \
      co-B        co-A
        |          |
        a -------- b

"co-" means complement (90° - x)

The five parts in order: \(a, b, \text{co-}A, \text{co-}c, \text{co-}B\)

The Two Rules

Rule 1: Sine of Middle = Product of Tangents of Adjacents

\[\sin(\text{middle}) = \tan(\text{adjacent}_1) \times \tan(\text{adjacent}_2)\]

Rule 2: Sine of Middle = Product of Cosines of Opposites

\[\sin(\text{middle}) = \cos(\text{opposite}_1) \times \cos(\text{opposite}_2)\]

The Ten Formulas

Applying Napier’s Rules generates all ten formulas for right spherical triangles:

Middle Part Formula

\(a\)

\(\sin a = \sin A \sin c = \tan b \cot B\)

\(b\)

\(\sin b = \sin B \sin c = \tan a \cot A\)

\(\text{co-}A\) (i.e., \(\cos A\))

\(\cos A = \cos a \sin B = \tan b \cot c\)

\(\text{co-}B\) (i.e., \(\cos B\))

\(\cos B = \cos b \sin A = \tan a \cot c\)

\(\text{co-}c\) (i.e., \(\cos c\))

\(\cos c = \cos a \cos b = \cot A \cot B\)

Critical Formulas for Celestial Navigation

Computing Altitude (Hc) and Azimuth (Zn)

In the celestial navigation triangle:

  • \(P\) = elevated pole (north or south)

  • \(Z\) = zenith

  • \(X\) = celestial body

With: * \(t\) = local hour angle (LHA) * \(\phi\) = observer’s latitude * \(d\) = declination of celestial body * \(H_c\) = computed altitude * \(Z\) = azimuth angle

Altitude:

\[\sin H_c = \sin \phi \sin d + \cos \phi \cos d \cos t\]

Azimuth angle:

\[\cos Z = \frac{\sin d - \sin \phi \sin H_c}{\cos \phi \cos H_c}\]

Or using the four-parts formula:

\[\tan Z = \frac{\sin t}{\cos \phi \tan d - \sin \phi \cos t}\]

Quadrantal Triangles

When one side equals 90°, the triangle is quadrantal. Napier’s Rules apply by substituting the co-functions.

For a quadrantal triangle with \(c = 90°\):

\[\cos C = -\cos A \cos B\]
\[\sin a = \sin A \sin C\]

Practical Application: Sight Reduction

Example: Computing Hc for Sun Observation

Given: * Latitude \(\phi\) = 33°45.0’N = 33.75° * Declination \(d\) = 15°20.5’N = 15.342° * LHA \(t\) = 45°30.0' = 45.5°

Step 1: Apply the altitude formula:

\[\sin H_c = \sin(33.75°)\sin(15.342°) + \cos(33.75°)\cos(15.342°)\cos(45.5°)\]
\[\sin H_c = (0.5554)(0.2645) + (0.8316)(0.9644)(0.7009)\]
\[\sin H_c = 0.1469 + 0.5621 = 0.7090\]
\[H_c = \arcsin(0.7090) = 45.17° = 45°10.2'\]

Great Circle Distance

The shortest path between two points on a sphere follows a great circle.

Haversine Formula

The most numerically stable formula for calculating great circle distance:

\[a = \sin^2\left(\frac{\Delta\phi}{2}\right) + \cos(\phi_1) \cdot \cos(\phi_2) \cdot \sin^2\left(\frac{\Delta\lambda}{2}\right)\]
\[c = 2 \cdot \arctan2\left(\sqrt{a}, \sqrt{1-a}\right)\]
\[d = R \cdot c\]

Where:

  • \(\phi_1, \phi_2\) = latitudes in radians

  • \(\lambda_1, \lambda_2\) = longitudes in radians

  • \(\Delta\phi = \phi_2 - \phi_1\)

  • \(\Delta\lambda = \lambda_2 - \lambda_1\)

  • \(R\) = Earth’s radius (6,371 km mean)

  • \(d\) = distance

Why Haversine?

The spherical law of cosines has numerical issues for small distances:

\[d = R \cdot \arccos\left(\sin(\phi_1)\sin(\phi_2) + \cos(\phi_1)\cos(\phi_2)\cos(\Delta\lambda)\right)\]

When \(\Delta\lambda\) is small, \(\cos(\Delta\lambda) \approx 1\), causing precision loss.

Haversine uses \(\sin^2\) functions which are stable near zero.

Example Calculation

LAX (33.9425°N, 118.4081°W) to JFK (40.6413°N, 73.7781°W)
import math

# Convert to radians
lat1 = math.radians(33.9425)
lon1 = math.radians(-118.4081)
lat2 = math.radians(40.6413)
lon2 = math.radians(-73.7781)

dlat = lat2 - lat1
dlon = lon2 - lon1

# Haversine
a = math.sin(dlat/2)**2 + math.cos(lat1) * math.cos(lat2) * math.sin(dlon/2)**2
c = 2 * math.atan2(math.sqrt(a), math.sqrt(1-a))
R = 6371  # km

distance = R * c
print(f"Distance: {distance:.2f} km")  # 3974.21 km

Bearing Calculations

Initial Bearing (Forward Azimuth)

The direction to follow from start to reach destination:

\[\theta = \arctan2\left(\sin(\Delta\lambda) \cdot \cos(\phi_2), \cos(\phi_1) \cdot \sin(\phi_2) - \sin(\phi_1) \cdot \cos(\phi_2) \cdot \cos(\Delta\lambda)\right)\]

Result in radians, convert to degrees and normalize to 0-360°.

Final Bearing (Reverse Azimuth)

The bearing when arriving at destination:

\[\theta_{final} = (\theta_{initial \text{ reversed}} + 180°) \mod 360°\]

Calculate initial bearing from destination to start, then add 180°.

Why Bearings Change

On a great circle route, the bearing constantly changes (except on meridians or equator).

      N
      │
  LAX → → → → → → → → JFK
  65° → 67° → 70° → ... → 90°

Initial bearing: ~66° (ENE)
Final bearing: ~90° (E)

Destination Point Calculation

Given start point, bearing, and distance, find destination:

\[\phi_2 = \arcsin\left(\sin(\phi_1) \cdot \cos\left(\frac{d}{R}\right) + \cos(\phi_1) \cdot \sin\left(\frac{d}{R}\right) \cdot \cos(\theta)\right)\]
\[\lambda_2 = \lambda_1 + \arctan2\left(\sin(\theta) \cdot \sin\left(\frac{d}{R}\right) \cdot \cos(\phi_1), \cos\left(\frac{d}{R}\right) - \sin(\phi_1) \cdot \sin(\phi_2)\right)\]

Midpoint Calculation

\[B_x = \cos(\phi_2) \cdot \cos(\Delta\lambda)\]
\[B_y = \cos(\phi_2) \cdot \sin(\Delta\lambda)\]
\[\phi_m = \arctan2\left(\sin(\phi_1) + \sin(\phi_2), \sqrt{(\cos(\phi_1) + B_x)^2 + B_y^2}\right)\]
\[\lambda_m = \lambda_1 + \arctan2(B_y, \cos(\phi_1) + B_x)\]

Rhumb Line vs Great Circle

Rhumb Line (Loxodrome)

A line that crosses all meridians at the same angle. Constant bearing, but NOT the shortest path. The rhumb line spirals toward the poles, never reaching them (except for cardinal bearings).

Rhumb Line Bearing

The constant bearing (course) of a rhumb line:

\[\theta = \arctan\left(\frac{\Delta\lambda}{\Delta\psi}\right)\]

Where \(\Delta\psi\) is the Mercator latitude difference:

\[\psi = \ln\left[\tan\left(\frac{\pi}{4} + \frac{\phi}{2}\right)\right]\]

Or equivalently (the inverse Gudermannian):

\[\psi = \sinh^{-1}(\tan\phi) = \tanh^{-1}(\sin\phi)\]

Rhumb Line Distance

\[d = \frac{\Delta\phi}{\cos\theta}\]

When bearing is 90° or 270° (east-west):

\[d = \Delta\lambda \cdot \cos\phi\]

Complete formula:

\[d = R \sqrt{\Delta\phi^2 + q^2 \Delta\lambda^2}\]

Where \(q\) is:

\[q = \begin{cases} \cos\phi & \text{if } \phi_1 = \phi_2 \\ \frac{\Delta\phi}{\Delta\psi} & \text{otherwise} \end{cases}\]

Great Circle vs Rhumb Line Comparison

New York to London
  • Great Circle: 5,570 km (initial bearing 051°, final bearing 108°)

  • Rhumb Line: 5,802 km (constant bearing 078°)

  • Difference: 232 km longer (4.2% increase in fuel/time)

LAX to Tokyo (extreme case)
  • Great Circle: 8,815 km (initial bearing 306°)

  • Rhumb Line: 9,653 km (constant bearing 276°)

  • Difference: 838 km longer (9.5% increase)

Comparison Table

Great Circle Rhumb Line

Path

Shortest distance

Constant bearing

Bearing

Changes continuously

Constant

Map projection

Curved on Mercator

Straight on Mercator

Computation

Spherical trig

Logarithmic (Mercator)

Use case

Long-distance travel

Short distances, simple navigation

Fuel efficiency

Optimal

Suboptimal (longer route)

Navigational ease

Complex (continuous adjustment)

Simple (hold one bearing)

When to Use Each

Scenario Recommended

Transoceanic flight

Great circle (fuel savings outweigh complexity)

Coastal sailing < 500 nm

Rhumb line (simpler, minimal penalty)

Artillery/surveying

Depends on range and required precision

Driving navigation

Roads don’t follow either!

Spherical Excess and Area

Spherical Excess

The sum of angles in a spherical triangle exceeds 180°. This excess is directly proportional to area.

\[E = A + B + C - 180°\]

Where \(E\) is the spherical excess in degrees.

The excess can also be computed via L’Huilier’s Theorem:

\[\tan\left(\frac{E}{4}\right) = \sqrt{\tan\left(\frac{s}{2}\right)\tan\left(\frac{s-a}{2}\right)\tan\left(\frac{s-b}{2}\right)\tan\left(\frac{s-c}{2}\right)}\]

Where \(s = \frac{a+b+c}{2}\) is the semi-perimeter.

Area of Spherical Triangle

\[\text{Area} = R^2 \cdot E_{\text{radians}}\]

Converting from degrees:

\[\text{Area} = R^2 \cdot E \cdot \frac{\pi}{180}\]
Example: Large Navigation Triangle

A spherical triangle with angles 120°, 100°, and 80°:

\[E = 120° + 100° + 80° - 180° = 120°\]
\[\text{Area} = (6371)^2 \times 120° \times \frac{\pi}{180} = 84,996,000 \text{ km}^2\]

That’s about 17% of Earth’s surface area!

Area of Spherical Polygon

For a polygon with \(n\) vertices:

\[\text{Area} = R^2 \left[\sum_{i=1}^{n} \theta_i - (n-2)\pi\right]\]

Where \(\theta_i\) are the interior angles in radians.

Vector Operations

Position Vector (ECEF)

Convert lat/lon to unit vector on sphere:

\[\vec{v} = \begin{bmatrix} \cos(\phi)\cos(\lambda) \\ \cos(\phi)\sin(\lambda) \\ \sin(\phi) \end{bmatrix}\]

Cross Product

For finding perpendicular vectors:

\[\vec{a} \times \vec{b} = \begin{bmatrix} a_y b_z - a_z b_y \\ a_z b_x - a_x b_z \\ a_x b_y - a_y b_x \end{bmatrix}\]

Dot Product

For finding angles between vectors:

\[\vec{a} \cdot \vec{b} = a_x b_x + a_y b_y + a_z b_z = |\vec{a}||\vec{b}|\cos(\theta)\]

Error Propagation in Navigation

Understanding position uncertainty is critical for military operations. A position is never known exactly—only to within some error bound.

Error Propagation Fundamentals

For a function \(f(x_1, x_2, \ldots, x_n)\) with independent variables having uncertainties \(\sigma_{x_1}, \sigma_{x_2}, \ldots\), the propagated uncertainty is:

\[\sigma_f = \sqrt{\sum_{i=1}^{n} \left(\frac{\partial f}{\partial x_i}\right)^2 \sigma_{x_i}^2}\]

Position Uncertainty Circle

For a 2D position with independent errors:

\[\sigma_r = \sqrt{\sigma_x^2 + \sigma_y^2}\]

The Circular Error Probable (CEP) is the radius containing 50% of positions:

\[\text{CEP} = 0.5887 \times (\sigma_x + \sigma_y) \quad \text{(for } \sigma_x \approx \sigma_y \text{)}\]

More precisely (Rayleigh distribution when \(\sigma_x = \sigma_y = \sigma\)):

\[\text{CEP} = 1.1774 \times \sigma\]

Military Error Standards

Standard Probability Definition

CEP (Circular Error Probable)

50%

Radius containing half of positions

SEP (Spherical Error Probable)

50%

3D sphere containing half of positions

2DRMS (2× Distance RMS)

95-98%

\(2\sqrt{\sigma_x^2 + \sigma_y^2}\)

R95

95%

Radius containing 95% of positions

Distance Error Propagation

For the Haversine formula, the error in computed distance depends on errors in input coordinates:

\[\sigma_d = \sqrt{\left(\frac{\partial d}{\partial \phi_1}\right)^2 \sigma_{\phi_1}^2 + \left(\frac{\partial d}{\partial \lambda_1}\right)^2 \sigma_{\lambda_1}^2 + \left(\frac{\partial d}{\partial \phi_2}\right)^2 \sigma_{\phi_2}^2 + \left(\frac{\partial d}{\partial \lambda_2}\right)^2 \sigma_{\lambda_2}^2}\]

For small distances, a simplified rule of thumb:

\[\sigma_d \approx R \sqrt{\sigma_{\phi_1}^2 + \sigma_{\phi_2}^2 + \cos^2\phi \left(\sigma_{\lambda_1}^2 + \sigma_{\lambda_2}^2\right)}\]

Bearing Error Propagation

The bearing from point 1 to point 2:

\[\sigma_\theta \approx \frac{\sigma_r}{d}\]

Where: * \(\sigma_r\) = cross-track position uncertainty * \(d\) = distance to target

At 1 km with 10m position error: \(\sigma_\theta \approx 10/1000 = 0.01 \text{ rad} \approx 0.57°\)

Dead Reckoning Error Accumulation

Dead reckoning errors accumulate with time/distance. For a leg of length \(L\):

\[\sigma_{\text{along}} = \sigma_v \cdot t + v \cdot \sigma_t\]
\[\sigma_{\text{cross}} = L \cdot \sigma_\theta\]

Where: * \(\sigma_v\) = speed error * \(\sigma_t\) = time error * \(\sigma_\theta\) = heading error (radians)

For multiple legs, errors combine:

\[\sigma_{\text{total}} = \sqrt{\sum_{i=1}^{n} \sigma_i^2} \quad \text{(independent errors)}\]

Error Ellipse

Position uncertainty is often an ellipse, not a circle:

\[\frac{(x - \mu_x)^2}{\sigma_x^2} + \frac{(y - \mu_y)^2}{\sigma_y^2} = k^2\]

Where \(k\) determines probability: * \(k = 1.0\) → 39.4% (1-sigma ellipse) * \(k = 2.0\) → 86.5% (2-sigma ellipse) * \(k = 2.45\) → 95% confidence ellipse * \(k = 3.0\) → 98.9% (3-sigma ellipse)

Rotated Error Ellipse

When errors are correlated, the ellipse is rotated:

\[\theta = \frac{1}{2} \arctan\left(\frac{2\rho\sigma_x\sigma_y}{\sigma_x^2 - \sigma_y^2}\right)\]

Where \(\rho\) is the correlation coefficient.

GPS Error Model

GPS position error follows a roughly Gaussian distribution:

Source Error (1σ) Notes

Ionospheric delay

2-5 m

Varies with solar activity

Tropospheric delay

0.5 m

Weather dependent

Satellite ephemeris

2 m

Broadcast vs precise

Receiver noise

0.3 m

Hardware dependent

Multipath

0.5-2 m

Environment dependent

PDOP-weighted total

3-10 m

Civilian L1 C/A

Position Dilution of Precision (PDOP):

\[\sigma_{\text{position}} = \text{UERE} \times \text{PDOP}\]

Where UERE = User Equivalent Range Error (combined single-satellite error)

Coriolis Effect (Long-Range Ballistics)

For long-range artillery and sniper fire, the Coriolis effect deflects projectiles:

Horizontal Deflection

\[\Delta x = \frac{2v \sin(\phi) \cdot t}{g} \cdot v_h\]

Simplified for horizontal fire:

\[\Delta x \approx \frac{v_h^2 \cdot t \cdot \Omega \sin\phi}{g}\]

Where: * \(\Omega\) = Earth’s angular velocity = 7.2921 × 10⁻⁵ rad/s * \(\phi\) = latitude * \(t\) = time of flight * \(v_h\) = horizontal velocity

Practical Rule (Sniper)

At 1000m, 45° latitude, typical deflection is 6-8 cm. At 2000m: 25-30 cm.

Northern Hemisphere: * Firing NORTH → bullet deflects RIGHT * Firing EAST → bullet deflects RIGHT * Firing SOUTH → bullet deflects RIGHT * Firing WEST → bullet deflects RIGHT

(All rightward due to Earth’s rotation; opposite in Southern Hemisphere)

Practical Calculations

CLI Implementation

haversine() {
    # Calculate great-circle distance using Haversine formula
    # Usage: haversine <lat1> <lon1> <lat2> <lon2> [unit]
    # Units: km (default), m, nm, mi
    # Example: haversine 33.9425 -118.4081 40.6413 -73.7781 km
    local lat1=$1 lon1=$2 lat2=$3 lon2=$4 unit=${5:-km}

    awk -v lat1="$lat1" -v lon1="$lon1" -v lat2="$lat2" -v lon2="$lon2" -v unit="$unit" '
    BEGIN {
        PI = 3.14159265358979323846

        # Convert to radians
        lat1_rad = lat1 * PI / 180
        lon1_rad = lon1 * PI / 180
        lat2_rad = lat2 * PI / 180
        lon2_rad = lon2 * PI / 180

        dlat = lat2_rad - lat1_rad
        dlon = lon2_rad - lon1_rad

        # Haversine formula
        a = sin(dlat/2)^2 + cos(lat1_rad) * cos(lat2_rad) * sin(dlon/2)^2
        c = 2 * atan2(sqrt(a), sqrt(1-a))

        # Select radius based on unit
        if (unit == "km") R = 6371.0088
        else if (unit == "m") R = 6371008.8
        else if (unit == "nm") R = 3440.065
        else if (unit == "mi") R = 3958.761
        else R = 6371.0088

        distance = R * c
        printf "%.4f\n", distance
    }'
}

Python Implementation

    """
    Convert geodetic coordinates to ECEF (Earth-Centered, Earth-Fixed)

    Uses WGS84 ellipsoid parameters.

    Formula:
        N = a / sqrt(1 - e² * sin²(φ))
        X = (N + h) * cos(φ) * cos(λ)
        Y = (N + h) * cos(φ) * sin(λ)
        Z = (N(1 - e²) + h) * sin(φ)
    """
    lat_rad = math.radians(point.lat)
    lon_rad = math.radians(point.lon)

    sin_lat = math.sin(lat_rad)
    cos_lat = math.cos(lat_rad)
    sin_lon = math.sin(lon_rad)
    cos_lon = math.cos(lon_rad)

    # Radius of curvature in prime vertical
    N = WGS84_A / math.sqrt(1 - WGS84_E2 * sin_lat ** 2)

    x = (N + point.height) * cos_lat * cos_lon
    y = (N + point.height) * cos_lat * sin_lon
    z = (N * (1 - WGS84_E2) + point.height) * sin_lat

    return ECEF(x, y, z)


def ecef_to_geographic(ecef: ECEF) -> GeoPoint:
    """
    Convert ECEF to geodetic coordinates using Bowring's iterative method
    """
    x, y, z = ecef.x, ecef.y, ecef.z

    # Longitude is straightforward
    lon = math.atan2(y, x)

    # Iterative solution for latitude
    p = math.sqrt(x**2 + y**2)
    lat = math.atan2(z, p * (1 - WGS84_E2))  # Initial approximation

    for _ in range(10):  # Usually converges in 2-3 iterations
        sin_lat = math.sin(lat)
        N = WGS84_A / math.sqrt(1 - WGS84_E2 * sin_lat**2)
        lat_new = math.atan2(z + WGS84_E2 * N * sin_lat, p)

Navigation Slide Rules

Before calculators, navigators used specialized slide rules.

Common Conversions

Operation Formula

Nautical miles → km

\(km = nm \times 1.852\)

Statute miles → km

\(km = mi \times 1.609\)

Knots → km/h

\(km/h = kts \times 1.852\)

1° latitude

≈ 111 km (60 nm)

1° longitude at equator

≈ 111 km

1° longitude at lat φ

\(\approx 111 \times \cos(\phi)\) km

Exercises

Drill 1: Distance Calculations

Calculate the great circle distance between:

  1. New York (40.7128°N, 74.0060°W) and London (51.5074°N, 0.1278°W)

  2. Tokyo (35.6762°N, 139.6503°E) and Sydney (33.8688°S, 151.2093°E)

  3. Los Angeles (34.0522°N, 118.2437°W) and Honolulu (21.3069°N, 157.8583°W)

Drill 2: Bearing Calculations

Calculate initial and final bearings:

  1. From San Francisco to Tokyo

  2. From London to New York

  3. From Sydney to Santiago, Chile

Drill 3: Destination Point

Starting from your current location:

  1. Travel 100 km on bearing 045° - where do you end up?

  2. Travel 500 nm on bearing 270° - where do you end up?

  3. Travel 1000 km on bearing 180° - where do you end up?

Quick Reference

Table 1. Essential Formulas
Formula Use

\(d = R \cdot \arccos(\sin\phi_1\sin\phi_2 + \cos\phi_1\cos\phi_2\cos\Delta\lambda)\)

Spherical law of cosines (distance)

Haversine (see above)

Numerically stable distance

\(\theta = \arctan2(\sin\Delta\lambda\cos\phi_2, \cos\phi_1\sin\phi_2 - \sin\phi_1\cos\phi_2\cos\Delta\lambda)\)

Initial bearing

\(1° lat \approx 111 km\)

Quick latitude distance

\(1° lon \approx 111 \cos(\phi) km\)

Quick longitude distance

Next

Continue to Time & Longitude to understand the relationship between time and position.