S D M X

CountryData
Using SDMX to exchange MDG data between national statistical offices and international agencies
MSIS 2014, 14-16 April 2014, Manila
Abdulla Gozalov
United Nations Statistics Division
Statistical Data & Metadata eXchange
“Set of technical standards and content-oriented
guidelines, together with an IT architecture and tools,
to be used for the efficient exchange and sharing of
statistical data and metadata.” (MCV 2009)
SDMX
ISO TS 17369
UNSD
World Bank
1
UNSD‐DfID
CountryData Project objectives
• Improve the coherence and clarity of dev’t indicators
- Improve coordination in the NSS
- Collate development data in 1 place/database
- Explain differences between intl. & nat. data
• Improve accessibility and visibility
- Make access to national data easier
- Draw attention to wider set of indicators
- Reduce data request
q
burden
• Enhance knowledge
- Strengthen IT support
- Use of latest IT software & practices
- Training & skills development
SDMX Background
• Perceived by some to be overly complex and difficult
to learn
– New tools, e.g. Eurostat’s SDMX-RI, greatly simplify
implementation and lower its cost.
– Proficiency in SDMX is still required.
• SDMX exchange established, for the most part,
between international organizations or advanced
countries and international organizations.
• Project gives the participants their first SDMX
experience with well-known and easy-to-use platform
and tools, such as DevInfo software or their own
existing dissemination systems.
2
CountryData SDMX Data Exchange Model
National Statistical Office
National
Repository DB
CountryData
Web Site
1. Publish
6. Publish
and Visualize
SDMX Dissemination
Site
CountryData DB
2. Register
International
MDG DB
5. Upload
4. Download
3. Notify
National
Registry
CountryData
Web Service
MDG Data Structure Definition
• Data Structure Definition (DSD) for MDG Indicators,
developed and approved by the Interagency and
Expert Group on MDGs Indicators (IAEG)
(IAEG), is used as
a basis for data exchange in the project.
• MDG code lists were extended, since non-MDG
indicators are also supported by the project.
• MDG Metadata
M t d t Structure
St
t
Definition,
D fi iti
available
il bl as a
draft, was adapted and used by the project for the
sharing of reference metadata.
3
SDMX Tools in DevInfo v7
• Mapping tool
– Visual interface that enables mapping between
D I f database
DevInfo
d t b
structures
t
t
and
d MDG Data
D t
Structure Definition
– Support for reference metadata in addition to data
– Key to lowering barrier to entry to SDMX
• Registry
– Structural metadata repository that supports
SDMX 2.0 and 2.1
– Supports SDMX subscriptions and notifications
– Has a SOAP-based API
– Implemented in 9 project countries
• Web portal developed for the project, part of the UNdata
platform
• A
Automatically
t
ti ll receives
i
and
d publishes
bli h data
d t from
f
countries’
ti ’
registries
– Provides re-dissemination and visualization of national data
– Facilitates, and displays the result of, analysis of
discrepancies between data from national sources and
international agencies.
• F
Focus on reference
f
metadata
t d t as the
th key
k to
t understanding
d
t di the
th
nature of national data and sources of discrepancies.
• Countries control what is published at CountryData
– Data is published by the countries without any intervention
by UNSD
4
Open Platform
• CountryData can accept data structured per MDG or
CountryData DSD.
– Thailand developed data exchange based on its advanced
data exchange and dissemination system, StatXChange,
and uses Metadata Technology’s Fusion Registry.
– Morocco has developed SDMX exchange using its own data
dissemination database, BDS.
• Mexico’s offer to share its indicators, compliant with
the MDG DSD,, was gladly
g
y accepted
p
and its data
published at the CountryData web site.
• Other countries have also expressed interest in
joining data exchange.
Next steps / Future
• Project finishes in January 2015
• Work with other international agencies to look at
data burden and how SDMX can assist
• Proposal for new project based on current project
architecture to support post-2015 agenda
• Greater calls for more disaggregated data
exchange; will investigate sub-national data
exchange using SDMX
5
Further UNdata Development
• Work is underway to implement SDMX API at UNdata
using SDMX Reference Infrastructure.
– MDG
MDG, C
CountryData,
t D t UNESCO
UNESCO, W
World
ld D
Development
l
t
Indicators, and World Population Prospects datasets are
already available.
• Development of DSDs for UNdata’s 34 datasets,
many of them highly disaggregated, is the biggest
challenge.
challenge
• MDG DSD continues to be promoted as an
international standard. It is expected that an
international group will be created to develop a DSD
for post-2015 indicators once those take shape.
UNdata API: How It Works
Mapping
M
i
Store
mappings
structures
mappings
?
data
UNdata
Web Service
6
Thank you for your attention
h
http://data.un.org/countrydata
//d
/
d
http://data.un.org
UNdata SDMX API
http://data.un.org/WS
h
http://data.un.org/SdmxBrowser
//d
/ d
http://unstats.un.org
7