Metis was developed to allow users to analyze regional and sub-regional multi-sector dynamics by providing a platform to harmonize and amalgamate data from different models and stakeholders operating at variable spatial scales. Metis is an open-source R package hosted on GitHub. Metis functions collectively allow users to compare, manipulate, and harmonize multi-sector data at user-specified spatial scales, and to identify and quantify sectoral inter-linkages. Each Metis function can also be used independently to support an array of other research applications, such as spatial analysis and data visualization.
Funding statement: This research was supported by the U.S. Department of Energy, Office of Science, as part of research in Multi-Sector Dynamics, Earth and Environmental System Modeling Program. The Pacific Northwest National Laboratory is operated for DOE by Battelle Memorial Institute under contract DE-AC05-76RL01830.
Important planning and decision making surrounding how to craft policies and allocate scarce financial resources to interventions (e.g., infrastructure) is often conducted at the regional (e.g., large river basin) or sub-regional (e.g., small river basin) scale. This regional planning has commonly been conducted in relative isolation by institutions focused on individual sectors (e.g., water resources management, electricity grid management, etc.) [1, 2, 3, 4]. This approach challenges the goal of holistic, cross-sectoral decision making, even when it is understood that a holistic approach would be beneficial.
This traditional planning paradigm is becoming decreasingly effective as a result of rapid interconnection among sectors, which exposes individual sectors to a wide array of forces across different spatiotemporal scales (from sub-regional to global). Understanding and modeling the interactions among energy, water, and land systems at regional and sub-regional levels presents substantial modeling and social science challenges [5, 6, 7].
There does not exist today a modeling platform that can effectively knit sub-regional energy, water, and land systems together at the regional level and connect them to national and global socioeconomic and climatic forces in an internally consistent, computationally efficient, data efficient, and decision-relevant way. Although multi-sectoral tools increasingly include representations of multiple sectors in a single analytical platform [8, 9, 10, 11, 12, 13], this approach has not been widely applied at regional and sub-regional scales . At the same time, a different line of research is attempting to link multiple detailed models together at regional levels . However, these linked-model methods are confined to research activities and have not proven tractable in operational decision-making contexts that require greater simplicity and computational efficiency for stakeholder engagement and exploration of uncertainty.
To address the challenges outlined above, the Metis model has been developed jointly by researchers from multiple institutions including the Pacific Northwest National Laboratory (PNNL), the University of Maryland (UMD), the Inter-American Development Bank (IDB), the Research Triangle Institute (RTI) and the Instituto Nacional del Agua (INA). Metis is a data organization and simulation platform that knits together multiple sectors within a single framework to facilitate analysis across sectors, including energy, water, land and socioeconomics, at any user-defined spatial and temporal scale of interest (e.g., provinces or river basins). Metis is designed to assemble, harmonize, and visualize data from different sectors in a consistent manner, and use that information to infer relationships between the sectors (e.g., how much water is used in the energy sector). These relationships, defined through an input-output matrix , can then be used to explore the cross-sector implications (i.e., nexus impacts) of alternative future policies or investments, such as hydropower or irrigation expansion. Similarly to multi-sectoral models, Metis represents all sectors with comparable detail and resolution, rather than focusing attention on representing particular sectors in more detail.
Data scarcity can be a significant constraint in regional and sub-regional planning studies seeking to evaluate multi-sector dynamics. Metis seeks to overcome this barrier by providing users with default data sets describing energy, water, and land supplies and demands for their specific region of interest. As shown on the left-hand side of Figure 1, the default data is built from outputs of the open-source Global Change Assessment Model (GCAM) and the ecosystem of modelling tools designed specifically to interact with GCAM . GCAM is used to capture long-term regional and global energy-water-land dynamics in response to drivers such as socioeconomic change, technological change, climatic change, and policy decisions. The broader GCAM ecosystem produces a globally consistent, downscaled set of gridded energy, water, and land data that Metis uses by default. The downscaling tools enable sub-regional planning issues to be linked to broader national and international dynamics. Metis aggregates the downscaled/gridded data to any spatial boundary, thus offering users an initial assessment of the energy, water, and land supplies and demands in their regions of interest. This initial assessment can then serve as the starting point that is refined over time as and when local data becomes available. Users can overwrite Metis’ default data sets at any time with these local data sets.
Figure 1 shows that the motivation behind developing Metis is to bridge the gap between sectoral models working at different spatial and temporal resolutions. Thus, as shown on the right-hand side of Figure 1, users can supplement global data sets with available outputs from models that capture specific sub-regional and sectoral details at finer resolution (e.g., water management or electricity planning models). While limited in scope to specific sectors, finer-scale models can nonetheless provide valuable input data to Metis. Metis integrates across tools by sharing data in standardized formats.
After aggregating data to relevant spatial and temporal scales, Metis identifies the relationships among sectors using linear input/output methods to establish intensity matrices. The intensity matrices represent the inter-sectoral links throughout the system. These linkages are reflected through intensity coefficients, which describes the commodity flows from one sector (e.g., electricity) that are required to produce a unit of output in another sector (e.g., water). These coefficients, which serve as the basis for calibration of the model, are then used to investigate impacts of changes in one sector on others. The sectoral interlinkages that exist in any given Metis application will depend upon the data used to populate the model. Examples include water demands for power plant cooling and hydropower; energy demands for water purification, transfers, and distribution; and both energy and water demands related to agriculture production and land-use change.
Metis is hosted on GitHub at https://github.com/JGCRI/metis, where users can find the latest version of the software and user documentation. Model details and features described in this publication pertain to the latest version of the model as of the date of this publication.
Metis is designed to be accessible to a range of stakeholders with varying expertise and goals related to multi-sector, multi-scale analyses. Various Metis functions can be used independently for different purposes, including visualization, charting, spatial aggregation, mapping, and inter-sectoral dynamics. Figure 2 summarizes key Metis modules and functions. Briefly, existing model capabilities include:
The following sub-sections walk users through the installation and example implementations of the key Metis functions shown in Figure 2.
Table 1 summarizes key functions in Metis. Each of these functions can be found in the directory ./metis/R. The main functions are then described with examples subsequently.
|metis.io.R||metis.io.R analyzes inter-sectoral dependencies using an input tables representing demands and supplies of different commodities and/or coefficient matrices representing inter-sectoral intensities. If not provided the function calculates linear inter-sectoral dependencies in the form of a coefficient matrix. Outputs include visualization of the inter-sectoral flows as Sankey diagrams as well as bubble plots representing flows and coefficient values.|
|metis.readgcam.R||This function is designed to interact specifically with GCAM outputs. The function processes GCAM outputs into .csv files by GCAM region, which can then be used as inputs to metis.chartsProcess.R|
|metis.chart.R||A charting function that allows quick and easy access to features like facets, labels and colors. The function is based on ggplot and returns a ggplot chart.|
|metis.chartsProcess.R||A charting function used to compare GCAM time series outputs across scenarios and regions. The function also creates diff plots with percentage and absolute differences from a given reference scenario.|
|metis.map.R||A mapping function used to plot raster and polygon data. The function uses the tmap package and returns a tmap object. Several maps can be combined by overlaying and underlaying using this function. Options allow for different color palettes, labels, and text-size. Visualization features include legend breaks that are freescale, kmeans, or equally divided to highlight different kinds of data.|
|metis.boundaries.R||Metis mapping function to plot shape file boundaries and surrounding regions for quick visualization of any region of interest.|
|metis.grid2poly.R||Function used to crop and aggregate gridded data by a given polygon shape file. If no grid is provided, the function can still be used to produce regional and subregional maps.|
|metis.mapsProcess.R||Metis mapping function used to compare across scenarios. The function produces diff maps with percentage and absolute differences from a given reference scenario.|
|metis.prepGrid.R||This function is designed to be used with specific open-source downscaling models (Xanthos , Demeter , and Tethys ) that downscale GCAM data to the grid level. The function takes outputs from these various models and processes them into the format required for providing input to the metis.mapsProcess.R function.|
|metis.assumptions.R||Contains all conversions and assumptions used in the model.|
|metis.colors.R||Collection of Metis color palettes. A list of palettes can be viewed in the function help file (?metis.colors). To view a particular palette: metis.colors(“pal_hot”).|
|metis.networkOrder.R||Determines the order in which to route water flows through the network of sub-regions, given a user-specified network connectivity matrix.|
|metis.waterBalance.R||Determines the natural flows from upstream sub-regions to downstream sub-regions via routing.|
metis.io.R is the Metis function used to analyze inter-sectoral intensities. metis.io.R uses an input/output methodology  to calculate linear intensity coefficient matrices. The function accepts a table of values giving flows of commodities labelled “supplySector” to any number of other sectors including the “supplySector” categories. The function then finds inter-sectoral links by analyzing the flows from given supply sectors to other supply sector categories within the region. Outputs of the function include tables, bubble charts and Sankey diagrams showing the intensity matrices and inter-sectoral links as shown in Figure 3. In Figure 3b the intensity matrix shows the intensity of inter-related sectors such as the “ag” sector which requires 0.97 units of “water” and 0.48 units of “elec” to produce 1 unit.
A key feature of metis is the ability to interact with different models. The core Metis model comes with the ability to connect directly with GCAM. This connection is made through the metis.readgcam.R function.
The model comes with an example GCAM database “.proj” file called Example_dataProj.proj, which lies in the directory ./metis/dataFiles/gcam/. GCAM produces output in the form of a database. The database contains outputs from various scenario runs. metis.readgcam.R uses the R package rgcam to establish a connection with the GCAM database and retrieves data based on “queries” provided in an “.xml” file. An example query file called metisQueries.xml is also provided in the same folder. Often scenario names in the model can be long and not appropriate for final figures. This function allows you to rename the scenarios as they are read in. Once the data has been extracted from a GCAM database it is saved in a “.proj” file. Reading data from the GCAM database can take a considerable amount of time depending on the number of scenarios it contains. The metis.readgcam.R function gives the option of directly providing a “.proj” file, which can be loaded directly, or using the “.proj” file from a previous run by setting the parameter reReadData to FALSE.
The metis.master.R file provides a step-by-step guide to connecting with GCAM model outputs. GCAM being a global multi-sectoral human-earth system model provides users with a large database of outputs which can be difficult to manipulate. The connection with Metis will allow users to easily sub-set relevant parameters of interest and compare across scenarios and time periods as well as visualize these results spatially. This functionality will assist planners and stakeholders to easily visualize and assess the regional and sub-regional implications of various policies, climate change, and different socio-economic and technological pathways on multiple resource sectors such as water, energy, and land.
metis.chart.R is the Metis charting software used in metis.chartsProcess.R. It allows users to create line, bar, bubble, and Sankey charts. The default settings maintain a consistent appearance across the metis products. metis.master.R provides examples to explore the metis.chart.R capabilities.
After running metis.chartsProcess.R several charts will be produced. These are saved in separate directories for each region, and within each directory Metis creates a sub-directory for each scenario as well as sub-directories for cross-scenario comparisons. In addition to the individual regional directories a directory called compareRegions contains inter-regional comparisons for each scenario as well as a cross-region/cross-scenario comparison. Tables with the data used for each figure are also provided in corresponding folders. metis.master.R provides examples for using the example data from the default GCAM database to show how different parameters across scenarios and regions can be compared using metis.chartsProcess.R. Some example figures are show in Figure 4.
metis.map.R is the metis charting software used in metis.mapsProcess.R and boundaries. It allows users to map outputs to shapefile polygons and raster files. The default settings maintain a consistent appearance across the metis products. metis.master.R provides examples to explore the metis.map.R capabilities.
metis.boundaries.R is used to create maps showing where a sub-region of interest lies in the greater region. For this option a boundary region should be defined. Natural Earth  geospatial files come with Metis and allow users to view country and state boundaries for the globe. Once metis.boundaries has been run a directory for “Maps” is created. The Maps directory will contain a “Boundary” directory which contains the boundary files defining each region and subregion. The function also shows how different grid cells sizes compare with the selected regions. This is useful to understand if the desired regional resolution is too fine for a particular grid size. Figure 5 shows example outputs of the function used to visualize different boundaries and how they overlap or intersect with other shapes and regions.
metis.grid2poly.R is used to aggregate gridded data to user-specified sub-regional spatial scales. metis.grid2poly.R accepts gridded data in the form of a table with latitude and longitude coordinates and a value for each point. Depending on the type of gridded data (volume or depth) the aggregation is done based on the part of the polygon that intersects with the grid cells, as shown in Figure 6. “Volume” here is defined as data such as volume of runoff or number of people in a grid cell. For this type of data the aggregate value for the polygon can be calculated as the sum of the values in each grid cell weighted by the percentage of area of that grid cell that overlaps with the polygon as shown in Method 1 in Figure 6. “Depth” on the other hand is data such as mm of water or water scarcity calculated as ratio. For this type of data the mean of the data weighted by the part of the area of the polygon within each grid cell is used to calculate the aggregate value for the polygon as shown in Method 2 Figure 6. Outputs from metis.grid2poly.R include tables with the original data by grid cell as well as new tables with data aggregated by the polygons from the shapefile provided.
metis.mapsProcess.R takes provided gridded and/or polygon data (which can be prepared using the suite of Metis functions defined previously) to produce maps as shown in the example in Figure 7. In addition, difference maps showing the absolute and percentage difference between a selected reference scenario and all other scenarios can also be produced. Each map is produced with three kinds of legends that allow the user to visualize spatial data in different ways: Freescale, Kmeans, and equal breaks. The color schemes for the plots are determined in metis.colors.R and can be adjusted by advanced users. Animations that show a map changing over time are also created for each type of map.
Metis has been designed to be modular and easily usable with outputs from any model or data product after the data has been processed into simple standardized Metis formats. A summary of the standard formats is provided below.
The function metis.charts.process.R can take as its input any “.csv” table or “R” table to produce all the charts described in the section “metis.charts.process.R” above to compare data across parameters, regions and scenarios. Any number of tables can be provided to the function as long as they contain the headers as shown in the blank template below (Table 2). This simple table can include data from any other model or raw data as collected from experts or local stakeholders.
Similarly, the function metis.maps.process.R is set up to accept tables for both polygons and rasters (gridded data with latitude and longitude co-ordinates) as described in the section “metis.maps.process.R”. These tables need to be in the format as shown in the example blank templates (Table 3 and Table 4). The polygon table must be accompanied with a shapefile that contains the region and sub-regions corresponding to the names listed in the table. As explained in the section “metis.boundaries.R”, Metis comes pre-loaded with global maps with national, state, provincial, and river basins boundary demarcations. For raster data (Table 4), sub-regions names should be replaced by latitude and longitude values. The function can be pointed to multiple tables and data as it is collected and Metis will produce corresponding visualizations to compare across scenarios, parameters, and time periods.
A key motivation behind allowing Metis functions to work with simple .csv tables is accessibility to a variety of stakeholders who may not be programmers or scientists. The templates presented in Tables 2, 3, and 4 can be easily understood and filled by anyone using any basic spreadsheet software. Data from various stakeholders can then easily be compared with outputs from other more detailed models. The accessibility to the input data in this simplified format accompanied by corresponding graphics from Metis facilitates consistent stakeholder engagement as data products and modeling scenarios evolve over the course of a project. Furthermore, the tables and data can be progressively filtered to hone-in on and identify key stakeholder and locally relevant parameters and nexus challenges.
As documented in this paper, Metis is a freely available open-source software and remains under constant development. Functions are designed to be highly modular and improvements and ideas for enhancements are welcome through the GitHub platform. Independent programmers and institutions are welcome (and encouraged) to fork versions of Metis and develop additional functionalities as needed. As described in the section “Using Metis with multiple models and data products”, Metis is designed to work with data tables in a simple standardized format. The function metis.readgcam.R is an example of a function designed to post-process data from outputs of the GCAM model. Users, working with other multi-sector or sector-specific models could easily write similar functions to post-process outputs from other models into the standardized templates presented above (Tables 2, 3, and 4).
metis.master.R in Metis’ home directory provides unit, integrated, and functional testing. The script allows users to run and test each individual function described briefly in this paper independently. Example data is provided with the installation of Metis so users can run the examples without creating their own data. Expected results for each function are posted on the webpage https://github.com/JGCRI/metis. The functions can also interact with outputs from each other and thus the entire script (metis.master.R) can be sourced as a functional test. At the end of metis.master.R a section on testing checks if the expected output was produced for each of the sections presented. A final line at the end of the script gives a summary of these test results.
Linux, Mac OSX, Windows 7 and up
R Version 3.1.5 and up
Note: packages marked with * may have additional requirements to be installed correctly on different operating systems. These are described in the detailed installation instructions in themetis.master.Rfile and include the packages devtools, rgdal, magick, and rgcam.
All contributors were listed as authors with corresponding roles and affiliations.
Persistent identifier: 10.5281/zenodo.3367767
Publisher: Zarrar Khan
Version published: v1.0.0
Date published: 14/08/19
Date published: 14/08/19
metis.master.R in Metis’ home directory provides users with detailed examples to test each key Metis function independently. The examples are illustrative and can be easily modified and adapted for different user needs. The model is hosted on GitHub and under active development with active support from the developers to address any bugs or issues that may arise.
The authors have no competing interests to declare.
Scanlon, B R, et al. 2017 The food-energy-water nexus: Transforming science for society. Water Resources Research, 53(5): 3550–3556. DOI: https://doi.org/10.1002/2017WR020889
White, D, Jones, J, Maciejewski, R, Aggarwal, R and Mascaro, G 2017 Stakeholder analysis for the food-energy-water nexus in phoenix, Arizona: Implications for nexus governance. Sustainability, 9(12): 2204. DOI: https://doi.org/10.3390/su9122204
Scott, C A, Kurian, M and Wescoat, J L 2015 The Water-Energy-Food Nexus: Enhancing Adaptive Capacity to Complex Global Challenges. In: Governing the Nexus, Kurian, M and Ardakanian, R (eds.), 15–38. Cham: Springer International Publishing. [Online]. Available: http://link.springer.com/10.1007/978-3-319-05747-7_2. [Accessed: 18-Dec-2019]. DOI: https://doi.org/10.1007/978-3-319-05747-7_2
Liu, J, et al. Sep. 2018 Nexus approaches to global sustainable development. Nature Sustainability, 1(9): 466–476. DOI: https://doi.org/10.1038/s41893-018-0135-8
Hibbard, K A and Janetos, A C 2013 The regional nature of global challenges: A need and strategy for integrated regional modeling. Climatic Change, 118(3–4): 565–577. DOI: https://doi.org/10.1007/s10584-012-0674-3
Rice, J S, Moss, R H, Runci, P J, Anderson, K L and Malone, E L 2012 Incorporating stakeholder decision support needs into an integrated regional Earth system model. Mitigation and adaptation strategies for global change, 17(7): 805–819. DOI: https://doi.org/10.1007/s11027-011-9345-3
Leung, R, Collins, B and Famiglietti, J 2012 Community Modeling and Long-Term Predictions of the Integrated Water Cycle. U.S. Department of Energy, Office of Science [Online]. Available: https://climatemodeling.science.energy.gov/sites/default/files/Water_Cycle_Report_LR.pdf.
Blanc, E, et al. Modeling US water resources under climate change, Earth’s Future, 2: 197–224. Received, 2013. DOI: https://doi.org/10.1002/2013EF000214
Hejazi, M I, et al. 2014 Integrated assessment of global water scarcity over the 21st century under multiple climate change mitigation policies. Hydrology and Earth System Sciences, 18(8): 2859–2883. DOI: https://doi.org/10.5194/hess-18-2859-2014
Hejazi, M, et al. 2014 Long-term global water projections using six socioeconomic scenarios in an integrated assessment modeling framework. Technological Forecasting and Social Change, 81: 205–226. DOI: https://doi.org/10.1016/j.techfore.2013.05.006
Schlosser, C A, et al. 2014 The future of global water stress: An integrated assessment. Earth’s Future, 2(8): 341–361. DOI: https://doi.org/10.1002/2014EF000238
Chaturvedi, V, et al. 2015 Climate mitigation policy implications for global irrigation water demand. Mitigation and adaptation strategies for global change, 20(3): 389–407. DOI: https://doi.org/10.1007/s11027-013-9497-4
Kim, S H, et al. 2016 Balancing global water availability and use at basin scale in an integrated assessment model. Climatic Change, 136(2): 217–231. DOI: https://doi.org/10.1007/s10584-016-1604-6
Vinca, A, et al. Jul. 2019 The Nexus Solutions Tool (NEST): An open platform for optimizing multi-scale energy-water-land system transformations. Geoscientific Model Development Discussions, 1–33. DOI: https://doi.org/10.5194/gmd-2019-134
Kraucunas, I, et al. 2015 Investigating the nexus of climate, energy, water, and land at decision-relevant scales: The Platform for Regional Integrated Modeling and Analysis (PRIMA). Climatic Change, 129(3–4): 573–588. DOI: https://doi.org/10.1007/s10584-014-1064-9
Miller, R E and Blair, P D 2009 Input–Output Analysis: Foundations and Extensions, 2nd ed. Cambridge: Cambridge University Press. [Online]. Available: http://ebooks.cambridge.org/ref/id/CBO9780511626982. [Accessed: 28-Jul-2019]. DOI: https://doi.org/10.1017/CBO9780511626982
Calvin, K, et al. Feb. 2019 GCAM v5.1: Representing the linkages between energy, water, land, climate, and economic systems. Geoscientific Model Development, 12(2): 677–698. DOI: https://doi.org/10.5194/gmd-12-677-2019
Li, X, et al. Sep. 2017 Xanthos – A Global Hydrologic Model. Journal of Open Research Software, 5 [Online]. Available: http://openresearchsoftware.metajnl.com/articles/10.5334/jors.181/. [Accessed: 26-Jul-2019]. DOI: https://doi.org/10.5334/jors.181
Le Page, Y, West, T O, Link, R and Patel, P Sep. 2016 Downscaling land use and land cover from the Global Change Assessment Model for coupling with Earth system models. Geoscientific Model Development, 9(9): 3055–3069. DOI: https://doi.org/10.5194/gmd-9-3055-2016
Li, X, et al. Feb. 2018 Tethys – A Python Package for Spatial and Temporal Downscaling of Global Water Withdrawals. Journal of Open Research Software, 6 [Online]. Available: http://openresearchsoftware.metajnl.com/articles/10.5334/jors.197/. [Accessed: 26-Jul-2019]. DOI: https://doi.org/10.5334/jors.197
Natural Earth. Downloads|Natural Earth. Natural Earth [Online]. Available: https://www.naturalearthdata.com/downloads/. [Accessed: 31-Jul-2019].