Skip to content

Implement currency data update workflow#61

Open
Wegatriespython wants to merge 5 commits intoIAMconsortium:mainfrom
Wegatriespython:feat/currency-data-update
Open

Implement currency data update workflow#61
Wegatriespython wants to merge 5 commits intoIAMconsortium:mainfrom
Wegatriespython:feat/currency-data-update

Conversation

@Wegatriespython
Copy link

This replaces the stubbed iam_units.update.currency() with a working generator and expands currency support from one hardcoded entry (EXC, EUR, 2005) to all five OECD Table 4 methods (EXC, EXCE, PPPGDP, PPPPRC, PPPP41) across five periods (2005, 2010, 2015, 2020, 2024) for EUR. This partly addresses #25.

The original stub in update.py specified three requirements for an implementation:

  • Use sdmx1 to query either the World Bank or OECD SDMX API.
  • Confirm the different data sources give the same results; if not, expose source selection. Cross-source validation against World Bank was tested manually for two data points (EXC and PPPGDP matched for DEU 2005), but the WDI SDMX endpoint was unreliable (403/404, timeouts), so WB is not wired in as a source. This can be revisited in a follow-up.
  • Write to a simple text format readable without dependencies, in iam_units/data/currency/, one file per (method, period).

The inline NB comments in currency.py noted two extensions: loading from file instead of embedded constants, and extending to cover other currencies. This PR addresses the first; EUR remains the only non-USD currency.

The current PR:

  • Implements currency() in the update submodule using sdmx1 to query OECD Table 4 (DSD_NAMAIN10@DF_TABLE4), following the pattern established for emissions data.
  • Writes plain-text data files to iam_units/data/currency/, committed as package data and loaded at runtime without any dependency on sdmx1.
  • Guards configure_currency() against silent wrong answers when called twice with different methods for the same currency/period pair — now raises ValueError. Same-method calls are idempotent.
  • Documents the update workflow in DEVELOPING.rst and the expanded capabilities in the README.

EUR rows are derived from the DEU series in Table 4. For exchange rates this is exact; for PPP methods the values are Germany-specific. This follows the approach in #60 which switched EUR deflators to Eurozone-wide values, but PPP is inherently country-level — the choice is documented in code and DEVELOPING.rst.

@khaeru
Copy link
Contributor

khaeru commented Mar 11, 2026

From the description this is great, thanks for the contribution!

One knee-jerk question prior to a real review: the 25 added data files each have 1 data point in them. Did you consider / would it be worse/better to follow an approach similar to the generation of emissions.py? IOW, generate a Python source file containing a data structure with all relevant data. This could be more compact.

@byersiiasa
Copy link

wow @Wegatriespython didn't expect you to address this within 2 hours. Thank you so much! ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants