PyLiPD User API
The following describes the main classes that makes up PyLiPD. Most users will only interface with the functionalies contained in these classes.
LiPD (pylipd.lipd.LiPD)
- class pylipd.lipd.LiPD(graph=None)[source]
The LiPD class describes a LiPD (Linked Paleo Data) object. It contains an RDF Graph which is serialization of the LiPD data into an RDF graph containing terms from the LiPD Ontology <http://linked.earth/Ontology/release/core/1.2.0/index-en.html> How to browse and query LiPD objects is described in a short example below.
Examples
In this example, we read an online LiPD file and convert it into a time series object dictionary.
from pylipd.lipd import LiPD lipd = LiPD() lipd.load(["https://lipdverse.org/data/LCf20b99dfe8d78840ca60dfb1f832b9ec/1_0_1//Nunalleq.Ledger.2018.lpd"]) ts_list = lipd.get_timeseries(lipd.get_all_dataset_names()) for dsname, tsos in ts_list.items(): for tso in tsos: if 'paleoData_variableName' in tso: print(dsname+': '+tso['paleoData_variableName']+': '+tso['archiveType'])
Loading 1 LiPD files
Loaded..
Extracting timeseries from dataset: Nunalleq.Ledger.2018 ... Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: precipitationUncertainty: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: ageMin: Archaeological Nunalleq.Ledger.2018: depth: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: precipitation: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: ageMedian: Archaeological Nunalleq.Ledger.2018: precipitationUncertainty: Archaeological Nunalleq.Ledger.2018: precipitation: Archaeological Nunalleq.Ledger.2018: precipitation: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: age: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: precipitation: Archaeological Nunalleq.Ledger.2018: precipitationUncertainty: Archaeological Nunalleq.Ledger.2018: precipitationUncertainty: Archaeological Nunalleq.Ledger.2018: precipitationUncertainty: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: temperatureUncertainty: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological Nunalleq.Ledger.2018: ageMax: Archaeological Nunalleq.Ledger.2018: precipitation: Archaeological Nunalleq.Ledger.2018: temperature: Archaeological
Methods
clear()Clears the graph
convert_lipd_dir_to_rdf(lipd_dir, rdf_file)Convert a directory containing LiPD files into a single RDF file (to be used for uploading to Knowledge Bases like GraphDB)
copy()Makes a copy of the object
create_lipd(dsname, lipdfile)Create LiPD file for a dataset
filter_by_archive_type(archiveType)Filters datasets to return a new LiPD object that only keeps datasets that have the specified archive type
filter_by_geo_bbox(lonMin, latMin, lonMax, ...)Filters datasets to return a new LiPD object that only keeps datasets that fall within the bounding box
get(dsnames)Gets dataset(s) from the graph and returns the popped LiPD object
Returns a list of all the unique archiveTypes present in the LiPD object
Get all Dataset ids
Get all Dataset Names
get_all_graph_ids()Get all Graph ids
get_all_locations([dsname])Return geographical coordinates for all the datasets.
Get a list of all possible distinct variableNames.
Returns a list of all variables in the graph
get_bibtex([remote, save, path, verbose])Get BibTeX for loaded datasets
Get a list of unique properties attached to a dataset.
get_ensemble_tables([dsname, ...])Gets ensemble tables from the LiPD graph
get_lipd(dsname)Get LiPD json for a dataset
Get all the properties associated with a model
get_timeseries(dsnames[, to_dataframe])Get Legacy LiPD like Time Series Object (tso)
get_timeseries_essentials([dsname, mode])Returns specific properties for timeseries: 'dataSetName', 'archiveType', 'geo_meanLat', 'geo_meanLon',
Get a list of variable properties that can be used for querying
load(lipdfiles[, parallel])Load LiPD files.
load_from_dir(dir_path[, parallel, cutoff])Load LiPD files from a directory
load_remote_datasets(dsnames)Loads remote datasets into cache if a remote endpoint is set
merge(rdf)Merges the current LiPD object with another LiPD object
pop(dsnames)Pops dataset(s) from the graph and returns the popped LiPD object
query(query[, remote, result])Once data is loaded into the graph (or remote endpoint set), one can make SparQL queries to the graph
remove(dsnames)Removes dataset(s) from the graph
serialize()Returns RDF quad serialization of the current combined Graph .
set_endpoint(endpoint)Sets a SparQL endpoint for a remote Knowledge Base (example: GraphDB)
to_lipd_series([parallel])Converts the LiPD object to a LiPDSeries object
update_remote_datasets(dsnames)Updates local LiPD Graph for datasets to remote endpoint
- convert_lipd_dir_to_rdf(lipd_dir, rdf_file, parallel=False)[source]
Convert a directory containing LiPD files into a single RDF file (to be used for uploading to Knowledge Bases like GraphDB)
- Parameters:
lipd_dir (str) – Path to the directory containing lipd files
rdf_file (str) – Path to the output rdf file
- create_lipd(dsname, lipdfile)[source]
Create LiPD file for a dataset
- Parameters:
dsname (str) – dataset id
lipdfile (str) – path to LiPD file
- Returns:
lipdjson – LiPD json
- Return type:
dict
Examples
from pylipd.lipd import LiPD # Load a local file lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", ]) dsname = lipd.get_all_dataset_names()[0] lipd.create_lipd(dsname, "test.lpd")
Loading 1 LiPD files
Loaded..
{'inCompilation1_': 'Ocean2k_v1.0.0', 'dataSetName': 'Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'inCompilation3_': 'PAGES2k_v2.1.0', 'originalDataURL': 'https://www.ncdc.noaa.gov/paleo/study/1866', 'changelog': {'notes': 'Starting the changelog', 'version': '1.0.0', 'timestamp': datetime.date(2022, 8, 23), 'curator': 'nicholas'}, 'maxYear': 1993.042, 'hasUrl': 'https://data.mint.isi.edu/files/lipd/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd', 'pub': [{'journal': 'Science', 'pages': '1511-1517', 'title': 'Variability in the El Nino-Southern Oscillation Through a Glacial-Interglacial Cycle', 'volume': 291.0, 'dataUrl': 'doi.org', 'citeKey': 'tudhope2001variabilityintheelninosou', 'author': [{'name': 'A. W. Tudhope'}], 'issue': 5508.0, 'doi': '10.1126/science.1057969', 'year': 2001.0, 'publisher': 'American Association for the Advancement of Science (AAAS)', 'identifier': []}, {'title': 'Tropical sea surface temperatures for the past four centuries reconstructed from coral archives', 'author': [{'name': 'Henry C. Wu'}, {'name': 'K. Halimeda Kilbourne'}, {'name': 'Jessica E. Tierney'}, {'name': 'Cyril Giry'}, {'name': 'Casey P. Saenger'}, {'name': 'Kevin J. Anchukaitis'}, {'name': 'Jens Zinke'}, {'name': 'Michael N. Evans'}, {'name': 'Nerilie J. Abram'}], 'citeKey': 'tierney2015tropicalseasurfacetempera', 'pages': '226-252', 'dataUrl': 'doi.org', 'publisher': 'Wiley-Blackwell', 'volume': 30.0, 'doi': '10.1002/2014PA002717', 'year': 2015.0, 'issue': 3.0, 'journal': 'Paleoceanography', 'identifier': []}, {'institution': 'World Data Center for Paleoclimatology', 'url': 'https://www.ncdc.noaa.gov/paleo/study/1866', 'author': [{'name': 'H. Kuhnert'}], 'citeKey': 'kuhnert2001httpswwwncdcnoaagovpaleostudy1866DataCitation', 'title': 'World Data Center for Paleoclimatology', 'urldate': 2001.0, 'identifier': []}], 'lipdVersion': 1.3, 'geo': {'geometry': {'coordinates': [145.8167, -5.2167, -2.2], 'type': 'Point'}, 'properties': {'type': 'http://linked.earth/ontology#Location', 'siteName': 'Madang Lagoon, Papua New Guinea', 'ocean': 'Pacific', 'pages2kRegion': 'Ocean'}}, 'paleoData': [{'measurementTable': [{'googleWorkSheetKey': 'ov9tjw6', 'tableName': 'Kuhnert', 'filename': 'Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.paleo1measurement1.csv', 'missingValue': 'NaN', 'columns': [{'wDSPaleoUrl': 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt', 'ocean2kID': 'PacificMadangTudhope2001', 'interpretation': [{'variableDetail': 'sea@surface', 'scope': 'climate', 'interpDirection': 'negative', 'seasonality': 'N/A (subannually resolved)', 'variable': 'T'}], 'measurementTableMD5': '793853407e414221c486d2e63b32dd87', 'hasResolution': {'units': 'AD', 'hasMeanValue': 0.25, 'hasMinValue': 0.25, 'hasMedianValue': 0.25, 'hasMaxValue': 0.25}, 'paleoDataTableName': 'measTable', 'hasMedianValue': -4.942, 'units': 'permil', 'measurementTableName': 'measurementTable1', 'pages2kID': 'Ocn_097', 'hasMeanValue': -4.9453, 'TSid': 'Ocean2kHR_140', 'proxy': 'd18O', 'sensorGenus': 'Porites', 'useInGlobalTemperatureAnalysis': True, 'proxyObservationType': 'd18O', 'notes': '; climateInterpretation_seasonality changed - was originally seasonal', 'hasMinValue': -5.515, 'inCompilationBeta': {'compilationVersion': '2_1_1', 'compilationName': 'Pages2kTemperature'}, 'iso2kUI': 'CO01TUNG01A', 'number': 1, 'qCCertification': 'KLD, NJA', 'variableName': 'd18O', 'hasMaxValue': -4.344, 'variableType': 'measured'}, {'measurementTableName': 'measurementTable1', 'inferredVariableType': 'Year', 'dataType': 'float', 'hasMinValue': 1880.792, 'units': 'AD', 'description': 'Year AD', 'number': 2, 'paleoDataTableName': 'measTable', 'hasResolution': {'hasMeanValue': 0.25, 'units': 'AD', 'hasMaxValue': 0.25, 'hasMedianValue': 0.25, 'hasMinValue': 0.25}, 'hasMeanValue': 1936.917, 'TSid': 'PYTDAS7AM1Y', 'wDSPaleoUrl': 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt', 'variableName': 'year', 'measurementTableMD5': '793853407e414221c486d2e63b32dd87', 'hasMaxValue': 1993.042, 'hasMedianValue': 1936.917, 'variableType': 'inferred'}]}]}], 'googleMetadataWorksheet': 'oruuxfm', 'studyName': 'Madang, Papua New Guinea oxygen isotope record 1880-1993', 'inCompilation2_': 'PAGES2k_v2.0.0', 'googleSpreadSheetKey': '1wf30P-s54OTBdLw4dyeaIN53VDoN0u_hOqIwvAeLxtc', 'createdBy': 'matlab', 'dataContributor': {'name': 'Wu KLD'}, 'googleDataURL': 'https://docs.google.com/spreadsheets/d/1wf30P-s54OTBdLw4dyeaIN53VDoN0u_hOqIwvAeLxtc', 'minYear': 1880.792, 'datasetId': 'm8yv2VgG97zJmSg3XhqQ', 'archiveType': 'coral'}
- filter_by_archive_type(archiveType)[source]
Filters datasets to return a new LiPD object that only keeps datasets that have the specified archive type
- Parameters:
archiveType (str) – The archive type to filter by
- Returns:
A new LiPD object that only contains datasets that have the specified archive type (regex)
- Return type:
Examples
pyLipd ships with existing datasets that can be loaded directly through the package. Let’s load the Pages2k sample datasets using this method.
from pylipd.utils.dataset import load_dir lipd = load_dir('Pages2k') Lfiltered = lipd.filter_by_archive_type('marine') Lfiltered.get_all_dataset_names()
Loading 16 LiPD files
Loaded..
['Ocn-AlboranSea436B.Nieto-Moreno.2013', 'Eur-CoastofPortugal.Abrantes.2011', 'Ocn-FeniDrift.Richter.2009']
- filter_by_geo_bbox(lonMin, latMin, lonMax, latMax)[source]
Filters datasets to return a new LiPD object that only keeps datasets that fall within the bounding box
- Parameters:
lonMin (float) – Minimum longitude
latMin (float) – Minimum latitude
lonMax (float) – Maximum longitude
latMax (float) – Maximum latitude
- Returns:
A new LiPD object that only contains datasets that fall within the bounding box
- Return type:
Examples
pyLipd ships with existing datasets that can be loaded directly through the package. Let’s load the Pages2k sample datasets using this method.
from pylipd.utils.dataset import load_dir lipd = load_dir() Lfiltered = lipd.filter_by_geo_bbox(0,25,50,50) Lfiltered.get_all_dataset_names()
Loading 16 LiPD files
Loaded..
['Ocn-SinaiPeninsula_RedSea.Moustafa.2000', 'Eur-SpannagelCave.Mangini.2005', 'Ocn-RedSea.Felis.2000', 'Eur-SpanishPyrenees.Dorado-Linan.2012', 'Eur-LakeSilvaplana.Trachsel.2010']
- get(dsnames)[source]
Gets dataset(s) from the graph and returns the popped LiPD object
- Parameters:
dsnames (str or list of str) – dataset name(s) to get.
- Returns:
LiPD object with the retrieved dataset(s)
- Return type:
Examples
from pylipd.lipd import LiPD # Load LiPD files from a local directory lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) all_datasets = lipd.get_all_dataset_names() print("Loaded datasets: " + str(all_datasets)) ds = lipd.get(all_datasets[0]) print("Got dataset: " + str(ds.get_all_dataset_names()))
Loading 2 LiPD files
Loaded.. Loaded datasets: ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007']
Got dataset: ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001']
- get_all_archiveTypes()[source]
Returns a list of all the unique archiveTypes present in the LiPD object
- Returns:
A list of archiveTypes
- Return type:
list
Examples
from pylipd.lipd import LiPD # Load Local files lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) print(lipd.get_all_archiveTypes())
Loading 2 LiPD files
Loaded.. ['coral', 'MarineSediment']
- get_all_dataset_ids()[source]
Get all Dataset ids
- Returns:
dsids (list)
A list of datasetnames
Examples
from pylipd.lipd import LiPD # Load local files lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) print(lipd.get_all_dataset_ids())
Loading 2 LiPD files
Loaded.. ['m8yv2VgG97zJmSg3XhqQ', 't0E8pOLYdyzmUspGZwbe']
- get_all_dataset_names()[source]
Get all Dataset Names
- Returns:
dsnames (list)
A list of datasetnames
Examples
from pylipd.lipd import LiPD # Load local files lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) print(lipd.get_all_dataset_names())
Loading 2 LiPD files
Loaded.. ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007']
- get_all_locations(dsname=None)[source]
Return geographical coordinates for all the datasets.
- Parameters:
dsname (str, optional) – The name of the dataset for which to return the timeseries information. The default is None.
- Returns:
df – A pandas dataframe returning the latitude, longitude and elevation for each dataset
- Return type:
pandas.DataFrame
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir('Pages2k') df = lipd.get_all_locations() print(df)
Loading 16 LiPD files
Loaded.. dataSetName geo_meanLat geo_meanLon \ 0 Eur-LakeSilvaplana.Trachsel.2010 46.5000 9.8000 1 Eur-SpanishPyrenees.Dorado-Linan.2012 42.5000 1.0000 2 Arc-Kongressvatnet.D'Andrea.2012 78.0217 13.9311 3 Eur-NorthernSpain.Martin-Chivelet.2011 42.9000 -3.5000 4 Ocn-SinaiPeninsula,RedSea.Moustafa.2000 27.8483 34.3100 5 Eur-Stockholm.Leijonhufvud.2009 59.3200 18.0600 6 Eur-NorthernScandinavia.Esper.2012 68.0000 25.0000 7 Ocn-RedSea.Felis.2000 27.8500 34.3200 8 Ocn-FeniDrift.Richter.2009 55.5000 -13.9000 9 Ant-WAIS-Divide.Severinghaus.2012 -79.4630 -112.1250 10 Eur-FinnishLakelands.Helama.2014 62.0000 28.3250 11 Ocn-AlboranSea436B.Nieto-Moreno.2013 36.2053 -4.3133 12 Eur-CoastofPortugal.Abrantes.2011 41.1000 -8.9000 13 Ocn-PedradeLume-CapeVerdeIslands.Moses.2006 16.7600 -22.8883 14 Eur-SpannagelCave.Mangini.2005 47.1000 11.6000 15 Asi-SourthAndMiddleUrals.Demezhko.2007 55.0000 59.5000 geo_meanElev 0 1791.0 1 1200.0 2 94.0 3 1250.0 4 -3.0 5 10.0 6 300.0 7 -6.0 8 -2543.0 9 1766.0 10 130.0 11 -1108.0 12 -80.0 13 -5.0 14 2347.0 15 1900.0
- get_all_variable_names()[source]
Get a list of all possible distinct variableNames. Useful for filtering and qeurying.
- Returns:
A list of unique variableName
- Return type:
list
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir('Pages2k') varName = lipd.get_all_variable_names() print(varName)
Loading 16 LiPD files
Loaded.. ['year', 'temperature', 'trsgi', 'Uk37', 'd18O', 'MXD', 'Mg_Ca.Mg/Ca', 'depth_top', 'notes', 'depth_bottom', 'uncertainty_temperature']
- get_all_variables()[source]
Returns a list of all variables in the graph
- Returns:
A dataframe of all variables in the graph with columns uri, varid, varname
- Return type:
pandas.DataFrame
Examples
from pylipd.lipd import LiPD lipd = LiPD() lipd.load([ "../examples/data/ODP846.Lawrence.2006.lpd" ]) df = lipd.get_all_variables() print(df)
Loading 1 LiPD files
Loaded.. uri TSID \ 0 http://linked.earth/lipd/paleo0model0ensemble0... PYTDW6AIJPW 1 http://linked.earth/lipd/chron0model0ensemble0... PYTGOFY4KZD 2 http://linked.earth/lipd/chron0model0summary0.... PYTDIEKUM44 3 http://linked.earth/lipd/paleo0measurement0.PY... PYT95DVDUU3 4 http://linked.earth/lipd/paleo0measurement1.PY... PYTYDOYFVYD 5 http://linked.earth/lipd/chron0model0summary0.... PYT4Y96QMUU 6 http://linked.earth/lipd/chron0model0summary0.... PYTI487BQDZ 7 http://linked.earth/lipd/paleo0measurement1.PY... PYTJZ4GLRYP 8 http://linked.earth/lipd/paleo0measurement0.PY... PYTM9N6HCQM 9 http://linked.earth/lipd/paleo0measurement1.PY... PYTS96EE0CB 10 http://linked.earth/lipd/paleo0measurement0.PY... PYT2ZB6MLZ9 11 http://linked.earth/lipd/paleo0measurement0.PY... PYTJ3PSH0LT 12 http://linked.earth/lipd/chron0measurement0.PY... PYT9CFQ4GK0 13 http://linked.earth/lipd/paleo0measurement0.PY... PYTKRFVW61B 14 http://linked.earth/lipd/paleo0model0ensemble0... PYTCHXB40SL 15 http://linked.earth/lipd/paleo0measurement1.PY... PYT19MC8WE2 16 http://linked.earth/lipd/chron0model0summary0.... PYT7DLYN7X4 17 http://linked.earth/lipd/paleo0measurement1.PY... PYTPQ0FJO1S 18 http://linked.earth/lipd/paleo0measurement1.PY... PYTTUPVG4K3 19 http://linked.earth/lipd/paleo0measurement0.PY... PYTXJB98403 20 http://linked.earth/lipd/paleo0measurement0.PY... PYTGO6NV72Y 21 http://linked.earth/lipd/chron0measurement0.PY... PYTTD7XCQGS 22 http://linked.earth/lipd/paleo0measurement1.PY... PYT3ZMI0BXW 23 http://linked.earth/lipd/chron0measurement0.PY... PYTLEHYPAYV 24 http://linked.earth/lipd/paleo0measurement0.PY... PYT10H23U2E 25 http://linked.earth/lipd/paleo0measurement1.PY... PYT68HYMYHH 26 http://linked.earth/lipd/chron0model0summary0.... PYTPWX0LH3I 27 http://linked.earth/lipd/chron0model0ensemble0... PYTUHE3XLGQ 28 http://linked.earth/lipd/paleo0measurement1.PY... PYTE5EC1JBW 29 http://linked.earth/lipd/paleo0measurement0.PY... PYT8BDSRW3H variableName 0 sst 1 depth 2 upper95 3 temp prahl 4 u. peregrina d18o.d18O 5 d180 6 lower95 7 sample label 8 ukprime37.Uk37Prime 9 depth 10 interval 11 site/hole 12 depth 13 depth 14 depth 15 c. wuellerstorfi d13c.d13C 16 median 17 depth comp 18 u. peregrina d13c.d13C 19 age 20 temp muller 21 age 22 c. wuellerstorfi d18o.d18O 23 d18o 24 c37 total.Alkenone 25 depth cr 26 depth 27 age 28 event 29 section
- get_bibtex(remote=True, save=True, path='mybiblio.bib', verbose=False)[source]
Get BibTeX for loaded datasets
- Parameters:
remote (bool) – (Optional) If set to True, will return the bibliography by checking against the DOI
save (bool) – (Optional) Whether to save the bibliography to a file
path (str) – (Optional) Path where to save the file
verbose (bool) – (Optional) Whether to print out on the console. Note that this option will turn on automatically if saving to a file fails.
- Returns:
bibs (list) – List of BiBTex entry
df (pandas.DataFrame) – Bibliography information in a Pandas DataFrame
Examples
from pylipd.lipd import LiPD # Fetch LiPD data from remote RDF Graph lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) print(lipd.get_bibtex(save=False))
Loading 2 LiPD files
Loaded..
Cannot find a matching record for the provided DOI, creating the entry manually
(['@article{Tierney_2015,\n\tdoi = {10.1002/2014pa002717},\n\turl = {https://doi.org/10.1002%2F2014pa002717},\n\tyear = 2015,\n\tmonth = {mar},\n\tpublisher = {American Geophysical Union ({AGU})},\n\tvolume = {30},\n\tnumber = {3},\n\tpages = {226--252},\n\tauthor = {Jessica E. Tierney and Nerilie J. Abram and Kevin J. Anchukaitis and Michael N. Evans and Cyril Giry and K. Halimeda Kilbourne and Casey P. Saenger and Henry C. Wu and Jens Zinke},\n\ttitle = {Tropical sea surface temperatures for the past four centuries reconstructed from coral archives},\n\tjournal = {Paleoceanography}\n}', '@article{Tudhope_2001,\n\tdoi = {10.1126/science.1057969},\n\turl = {https://doi.org/10.1126%2Fscience.1057969},\n\tyear = 2001,\n\tmonth = {feb},\n\tpublisher = {American Association for the Advancement of Science ({AAAS})},\n\tvolume = {291},\n\tnumber = {5508},\n\tpages = {1511--1517},\n\tauthor = {Alexander W. Tudhope and Colin P. Chilcott and Malcolm T. McCulloch and Edward R. Cook and John Chappell and Robert M. Ellam and David W. Lea and Janice M. Lough and Graham B. Shimmield},\n\ttitle = {Variability in the El Ni{\\~{n}}o-Southern Oscillation Through a Glacial-Interglacial Cycle},\n\tjournal = {Science}\n}', '@misc{kuhnert2001httpswwwncdcnoaagovpaleostudy1866DataCitation,\n author = "H. Kuhnert",\n title = "World Data Center for Paleoclimatology",\n institution = "World Data Center for Paleoclimatology",\n url = "https://www.ncdc.noaa.gov/paleo/study/1866"\n}\n', '@article{Stott_2007,\n\tdoi = {10.1029/2006pa001379},\n\turl = {https://doi.org/10.1029%2F2006pa001379},\n\tyear = 2007,\n\tmonth = {feb},\n\tpublisher = {American Geophysical Union ({AGU})},\n\tvolume = {22},\n\tnumber = {1},\n\tpages = {n/a--n/a},\n\tauthor = {Lowell D. Stott},\n\ttitle = {Comment on {\\textquotedblleft}Anomalous radiocarbon ages for foraminifera shells{\\textquotedblright} by W. Broecker et al.: A correction to the western tropical Pacific {MD}9821-81 record},\n\tjournal = {Paleoceanography}\n}', '@article{Stott_2007,\n\tdoi = {10.1126/science.1143791},\n\turl = {https://doi.org/10.1126%2Fscience.1143791},\n\tyear = 2007,\n\tmonth = {oct},\n\tpublisher = {American Association for the Advancement of Science ({AAAS})},\n\tvolume = {318},\n\tnumber = {5849},\n\tpages = {435--438},\n\tauthor = {Lowell Stott and Axel Timmermann and Robert Thunell},\n\ttitle = {Southern Hemisphere and Deep-Sea Warming Led Deglacial Atmospheric {CO}\n\t\t $\\less$sub$\\greater$2$\\less$/sub$\\greater$\n\t\t Rise and Tropical Warming},\n\tjournal = {Science}\n}'], dsname \ 0 Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001 1 Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001 2 Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001 3 MD98_2181.Stott.2007 4 MD98_2181.Stott.2007 title \ 0 Tropical sea surface temperatures for the past... 1 Variability in the El Nino-Southern Oscillatio... 2 World Data Center for Paleoclimatology 3 None 4 Southern Hemisphere and deep-sea warming led d... authors doi \ 0 Henry C. Wu and Casey P. Saenger and K. Halime... 10.1002/2014PA002717 1 A. W. Tudhope 10.1126/science.1057969 2 H. Kuhnert None 3 10.1029/2006PA001379 4 R. Thunell and A. Timmermann and L. Stott 10.1126/science.1143791 pubyear year journal volume issue pages \ 0 None 2015.0 Paleoceanography 30.0 3.0 226-252 1 None 2001.0 Science 291.0 5508.0 1511-1517 2 None NaN None NaN NaN None 3 None NaN None NaN NaN None 4 None 2007.0 Science 318.0 5849.0 435 438 type publisher report \ 0 journal-article Wiley-Blackwell None 1 journal-article American Association for the Advancement of Sc... None 2 dataCitation None None 3 None None None 4 None None None citeKey edition \ 0 tierney2015tropicalseasurfacetempera None 1 tudhope2001variabilityintheelninosou None 2 kuhnert2001httpswwwncdcnoaagovpaleostudy1866Da... None 3 None None 4 WMGAVB7S None institution url \ 0 None None 1 None None 2 World Data Center for Paleoclimatology None 3 None None 4 None None url2 0 None 1 None 2 https://www.ncdc.noaa.gov/paleo/study/1866 3 None 4 None )
- get_dataset_properties()[source]
Get a list of unique properties attached to a dataset.
Note: Some properties will return another object (e.g., ‘publishedIn’ will give you a Publication object with its own properties) Note: Not all datasets will have the same available properties (i.e., not filled in by a user)
- Returns:
clean_list – A list of avialable properties that can queried
- Return type:
list
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir(name='Pages2k') dataset_properties = lipd.get_dataset_properties() print(dataset_properties)
Loading 16 LiPD files
Loaded.. ['publishedIn', 'minYear', 'lipdVersion', 'includesPaleoData', 'collectedFrom', 'createdBy', 'proxyArchiveType', 'name', 'hasSpreadsheetLink', 'hasUrl', 'hasChangeLog', 'googleMetadataWorksheet', 'datasetId', 'googleDataURL', 'hasLink', 'type', 'maxYear', 'inCompilation2_', 'inCompilation1_', 'inCompilation3_', 'studyName', 'author', 'fundedBy', 'notes', 'contributor']
- get_ensemble_tables(dsname=None, ensembleVarName=None, ensembleDepthVarName='depth')[source]
Gets ensemble tables from the LiPD graph
- Parameters:
dsname (str) – The name of the dataset if you wish to analyse one at a time (Set to “.*” to match all datasets with a common root)
ensembleVarName (None or str) – ensemble variable name. Default is None, which searches for names that contain “year” or “age” (Set to “.*” to match all ensemble variable names)
ensembleDepthVarName (str) – ensemble depth variable name. Default is ‘depth’ (Set to “.*” to match all ensemble depth variable names)
- Returns:
ensemble_tables – A dataframe containing the ensemble tables
- Return type:
dataframe
Examples
from pylipd.lipd import LiPD lipd = LiPD() lipd.load([ "../examples/data/ODP846.Lawrence.2006.lpd" ]) all_datasets = lipd.get_all_dataset_names() print("Loaded datasets: " + str(all_datasets)) ens_df = lipd.get_ensemble_tables( ensembleVarName="age", ensembleDepthVarName="depth" ) print(ens_df)
Loading 1 LiPD files
Loaded.. Loaded datasets: ['ODP846.Lawrence.2006']
datasetName ensembleTable \ 0 ODP846.Lawrence.2006 http://linked.earth/lipd/chron0model0ensemble0 ensembleVariableName ensembleVariableValues \ 0 age [[4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0,... ensembleVariableUnits ensembleDepthName \ 0 kyr BP depth ensembleDepthValues ensembleDepthUnits notes \ 0 [0.12, 0.23, 0.33, 0.43, 0.53, 0.63, 0.73, 0.8... m None methodobj methods 0 None None
- get_lipd(dsname)[source]
Get LiPD json for a dataset
- Parameters:
dsname (str) – dataset id
- Returns:
lipdjson – LiPD json
- Return type:
dict
Examples
from pylipd.lipd import LiPD # Load a local LiPD file lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", ]) lipd_json = lipd.get_lipd(lipd.get_all_dataset_names()[0]) print(lipd_json)
Loading 1 LiPD files
Loaded.. {'inCompilation1_': 'Ocean2k_v1.0.0', 'pub': [{'year': 2001.0, 'title': 'Variability in the El Nino-Southern Oscillation Through a Glacial-Interglacial Cycle', 'dataUrl': 'doi.org', 'citeKey': 'tudhope2001variabilityintheelninosou', 'journal': 'Science', 'author': [{'name': 'A. W. Tudhope'}], 'volume': 291.0, 'issue': 5508.0, 'publisher': 'American Association for the Advancement of Science (AAAS)', 'doi': '10.1126/science.1057969', 'pages': '1511-1517', 'identifier': []}, {'citeKey': 'kuhnert2001httpswwwncdcnoaagovpaleostudy1866DataCitation', 'urldate': 2001.0, 'institution': 'World Data Center for Paleoclimatology', 'author': [{'name': 'H. Kuhnert'}], 'url': 'https://www.ncdc.noaa.gov/paleo/study/1866', 'title': 'World Data Center for Paleoclimatology', 'identifier': []}, {'author': [{'name': 'Henry C. Wu'}, {'name': 'K. Halimeda Kilbourne'}, {'name': 'Jessica E. Tierney'}, {'name': 'Cyril Giry'}, {'name': 'Casey P. Saenger'}, {'name': 'Kevin J. Anchukaitis'}, {'name': 'Jens Zinke'}, {'name': 'Michael N. Evans'}, {'name': 'Nerilie J. Abram'}], 'citeKey': 'tierney2015tropicalseasurfacetempera', 'year': 2015.0, 'title': 'Tropical sea surface temperatures for the past four centuries reconstructed from coral archives', 'journal': 'Paleoceanography', 'issue': 3.0, 'publisher': 'Wiley-Blackwell', 'doi': '10.1002/2014PA002717', 'volume': 30.0, 'pages': '226-252', 'dataUrl': 'doi.org', 'identifier': []}], 'dataSetName': 'Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'inCompilation3_': 'PAGES2k_v2.1.0', 'originalDataURL': 'https://www.ncdc.noaa.gov/paleo/study/1866', 'changelog': {'notes': 'Starting the changelog', 'version': '1.0.0', 'timestamp': datetime.date(2022, 8, 23), 'curator': 'nicholas'}, 'maxYear': 1993.042, 'hasUrl': 'https://data.mint.isi.edu/files/lipd/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd', 'lipdVersion': 1.3, 'geo': {'geometry': {'coordinates': [145.8167, -5.2167, -2.2], 'type': 'Point'}, 'properties': {'type': 'http://linked.earth/ontology#Location', 'siteName': 'Madang Lagoon, Papua New Guinea', 'ocean': 'Pacific', 'pages2kRegion': 'Ocean'}}, 'paleoData': [{'measurementTable': [{'googleWorkSheetKey': 'ov9tjw6', 'tableName': 'Kuhnert', 'filename': 'Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.paleo1measurement1.csv', 'missingValue': 'NaN', 'columns': [{'wDSPaleoUrl': 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt', 'ocean2kID': 'PacificMadangTudhope2001', 'interpretation': [{'variableDetail': 'sea@surface', 'scope': 'climate', 'interpDirection': 'negative', 'seasonality': 'N/A (subannually resolved)', 'variable': 'T'}], 'measurementTableMD5': '793853407e414221c486d2e63b32dd87', 'hasResolution': {'units': 'AD', 'hasMeanValue': 0.25, 'hasMinValue': 0.25, 'hasMedianValue': 0.25, 'hasMaxValue': 0.25}, 'paleoDataTableName': 'measTable', 'hasMedianValue': -4.942, 'units': 'permil', 'measurementTableName': 'measurementTable1', 'pages2kID': 'Ocn_097', 'hasMeanValue': -4.9453, 'TSid': 'Ocean2kHR_140', 'proxy': 'd18O', 'sensorGenus': 'Porites', 'useInGlobalTemperatureAnalysis': True, 'inCompilationBeta': {'compilationName': 'Pages2kTemperature', 'compilationVersion': '2_1_1'}, 'proxyObservationType': 'd18O', 'notes': '; climateInterpretation_seasonality changed - was originally seasonal', 'hasMinValue': -5.515, 'iso2kUI': 'CO01TUNG01A', 'number': 1, 'qCCertification': 'KLD, NJA', 'variableName': 'd18O', 'hasMaxValue': -4.344, 'variableType': 'measured', 'values': [-4.827, -4.786, -4.693, -4.852, -4.991, -4.904, -4.855, -4.862, -4.856, -4.947, -5.005, -5.298, -5.196, -5.298, -5.106, -5.375, -5.169, -5.083, -4.996, -5.027, -4.846, -4.646, -4.589, -4.972, -4.917, -4.795, -4.759, -5.301, -5.12, -5.086, -5.103, -5.244, -5.186, -5.059, -4.971, -5.356, -5.206, -4.885, -4.756, -4.959, -4.812, -4.667, -4.494, -5.117, -5.189, -5.133, -5.081, -5.165, -5.049, -4.883, -4.839, -5.103, -5.083, -4.96, -4.921, -5.204, -5.082, -5.133, -5.026, -5.334, -5.129, -4.889, -4.855, -5.214, -5.003, -4.842, -4.864, -5.038, -4.878, -5.027, -5.181, -5.515, -5.334, -5.06, -4.958, -5.268, -5.228, -5.136, -5.123, -5.304, -5.072, -4.748, -4.785, -5.234, -5.164, -5.053, -5.03, -5.188, -4.931, -4.99, -5.142, -5.216, -5.09, -4.865, -4.894, -5.041, -5.037, -5.064, -5.11, -5.291, -5.198, -5.242, -5.297, -5.492, -5.382, -5.087, -5.009, -5.282, -4.831, -4.524, -4.556, -5.071, -4.995, -4.958, -4.991, -5.08, -4.942, -4.908, -4.848, -4.993, -4.964, -5.031, -5.02, -5.245, -5.157, -5.041, -5.19, -5.39, -5.244, -5.072, -5.082, -5.362, -5.148, -4.889, -4.942, -4.895, -4.995, -4.924, -4.962, -5.0, -4.912, -4.774, -4.87, -5.104, -5.01, -4.996, -5.031, -5.112, -4.962, -5.003, -4.886, -5.095, -5.409, -4.992, -4.859, -5.151, -5.087, -5.031, -5.02, -5.355, -5.148, -5.13, -5.106, -5.228, -5.05, -4.868, -4.854, -4.937, -4.903, -4.902, -4.821, -4.963, -4.792, -4.886, -4.891, -5.146, -4.912, -4.856, -4.771, -4.964, -4.866, -4.909, -5.07, -5.459, -5.246, -4.871, -4.847, -5.086, -4.91, -4.912, -4.991, -5.203, -5.149, -4.957, -4.979, -5.232, -5.087, -5.072, -5.013, -5.329, -5.239, -5.016, -5.016, -5.079, -4.87, -4.786, -4.385, -4.525, -4.707, -4.607, -4.403, -4.607, -4.773, -4.846, -4.832, -4.925, -4.677, -4.487, -4.495, -4.597, -4.594, -4.629, -4.582, -4.832, -4.836, -4.687, -4.644, -4.967, -4.739, -4.803, -4.786, -5.133, -4.839, -4.899, -4.813, -4.973, -4.913, -5.002, -4.904, -5.114, -4.917, -4.886, -4.72, -4.926, -4.874, -4.677, -4.601, -4.924, -5.175, -4.906, -4.725, -5.135, -4.907, -4.829, -4.79, -5.185, -5.123, -4.988, -5.104, -5.33, -5.185, -5.158, -5.06, -5.282, -5.163, -5.06, -4.911, -5.03, -5.049, -4.688, -4.775, -5.055, -4.936, -4.807, -4.774, -5.162, -5.014, -4.975, -4.65, -4.919, -5.268, -4.892, -4.984, -5.139, -5.146, -4.998, -4.875, -5.035, -5.149, -5.123, -4.942, -5.108, -5.254, -4.856, -4.766, -5.051, -5.097, -4.715, -4.613, -4.786, -5.022, -4.986, -4.899, -4.96, -4.779, -4.897, -5.019, -5.453, -5.06, -4.788, -4.659, -4.767, -4.79, -4.378, -4.344, -4.727, -4.903, -4.875, -4.756, -4.988, -5.185, -4.943, -4.816, -4.839, -4.795, -4.747, -4.636, -4.753, -4.796, -4.716, -4.636, -4.775, -4.845, -4.809, -4.832, -5.013, -5.084, -4.909, -4.94, -5.031, -4.735, -4.625, -4.703, -4.933, -4.787, -4.808, -4.824, -5.266, -4.987, -4.634, -4.786, -5.08, -4.978, -4.988, -4.774, -5.006, -5.014, -4.866, -4.767, -4.722, -4.546, -4.359, -4.582, -5.062, -5.27, -5.077, -5.174, -5.2, -4.911, -4.96, -4.89, -5.115, -4.926, -4.839, -4.866, -5.211, -5.128, -5.037, -4.995, -5.131, -5.041, -4.969, -4.949, -5.052, -4.941, -4.671, -4.616, -5.11, -5.048, -4.751, -4.634, -5.052, -4.846, -4.742, -4.741, -4.903, -4.947, -4.848, -4.877, -4.886, -4.878, -4.956, -4.681, -4.941, -4.83, -5.185, -5.012, -4.967, -4.924, -4.77, -4.612, -4.957, -5.018, -4.97, -4.792, -4.788, -4.696, -4.525, -4.527, -4.721, -4.691, -4.853, -4.788, -4.931, -4.912, -4.954, -5.027, -4.937, -4.718, -4.512, -4.494, -4.675, -4.651, -4.666, -4.64, -4.849, -4.888, -4.833, -4.803, -4.863, -4.915, -4.733, -4.792, -4.872, -5.023, -4.923, -4.792, -4.906, -4.94, -4.801]}, {'measurementTableName': 'measurementTable1', 'inferredVariableType': 'Year', 'dataType': 'float', 'hasMinValue': 1880.792, 'units': 'AD', 'description': 'Year AD', 'number': 2, 'paleoDataTableName': 'measTable', 'hasResolution': {'hasMeanValue': 0.25, 'units': 'AD', 'hasMaxValue': 0.25, 'hasMedianValue': 0.25, 'hasMinValue': 0.25}, 'hasMeanValue': 1936.917, 'TSid': 'PYTDAS7AM1Y', 'wDSPaleoUrl': 'https://www1.ncdc.noaa.gov/pub/data/paleo/pages2k/pages2k-temperature-v2-2017/data-version-2.0.0/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.txt', 'variableName': 'year', 'measurementTableMD5': '793853407e414221c486d2e63b32dd87', 'hasMaxValue': 1993.042, 'hasMedianValue': 1936.917, 'variableType': 'inferred', 'values': [1993.042, 1992.792, 1992.542, 1992.292, 1992.042, 1991.792, 1991.542, 1991.292, 1991.042, 1990.792, 1990.542, 1990.292, 1990.042, 1989.792, 1989.542, 1989.292, 1989.042, 1988.792, 1988.542, 1988.292, 1988.042, 1987.792, 1987.542, 1987.292, 1987.042, 1986.792, 1986.542, 1986.292, 1986.042, 1985.792, 1985.542, 1985.292, 1985.042, 1984.792, 1984.542, 1984.292, 1984.042, 1983.792, 1983.542, 1983.292, 1983.042, 1982.792, 1982.542, 1982.292, 1982.042, 1981.792, 1981.542, 1981.292, 1981.042, 1980.792, 1980.542, 1980.292, 1980.042, 1979.792, 1979.542, 1979.292, 1979.042, 1978.792, 1978.542, 1978.292, 1978.042, 1977.792, 1977.542, 1977.292, 1977.042, 1976.792, 1976.542, 1976.292, 1976.042, 1975.792, 1975.542, 1975.292, 1975.042, 1974.792, 1974.542, 1974.292, 1974.042, 1973.792, 1973.542, 1973.292, 1973.042, 1972.792, 1972.542, 1972.292, 1972.042, 1971.792, 1971.542, 1971.292, 1971.042, 1970.792, 1970.542, 1970.292, 1970.042, 1969.792, 1969.542, 1969.292, 1969.042, 1968.792, 1968.542, 1968.292, 1968.042, 1967.792, 1967.542, 1967.292, 1967.042, 1966.792, 1966.542, 1966.292, 1966.042, 1965.792, 1965.542, 1965.292, 1965.042, 1964.792, 1964.542, 1964.292, 1964.042, 1963.792, 1963.542, 1963.292, 1963.042, 1962.792, 1962.542, 1962.292, 1962.042, 1961.792, 1961.542, 1961.292, 1961.042, 1960.792, 1960.542, 1960.292, 1960.042, 1959.792, 1959.542, 1959.292, 1959.042, 1958.792, 1958.542, 1958.292, 1958.042, 1957.792, 1957.542, 1957.292, 1957.042, 1956.792, 1956.542, 1956.292, 1956.042, 1955.792, 1955.542, 1955.292, 1955.042, 1954.792, 1954.542, 1954.292, 1954.042, 1953.792, 1953.542, 1953.292, 1953.042, 1952.792, 1952.542, 1952.292, 1952.042, 1951.792, 1951.542, 1951.292, 1951.042, 1950.792, 1950.542, 1950.292, 1950.042, 1949.792, 1949.542, 1949.292, 1949.042, 1948.792, 1948.542, 1948.292, 1948.042, 1947.792, 1947.542, 1947.292, 1947.042, 1946.792, 1946.542, 1946.292, 1946.042, 1945.792, 1945.542, 1945.292, 1945.042, 1944.792, 1944.542, 1944.292, 1944.042, 1943.792, 1943.542, 1943.292, 1943.042, 1942.792, 1942.542, 1942.292, 1942.042, 1941.792, 1941.542, 1941.292, 1941.042, 1940.792, 1940.542, 1940.292, 1940.042, 1939.792, 1939.542, 1939.292, 1939.042, 1938.792, 1938.542, 1938.292, 1938.042, 1937.792, 1937.542, 1937.292, 1937.042, 1936.792, 1936.542, 1936.292, 1936.042, 1935.792, 1935.542, 1935.292, 1935.042, 1934.792, 1934.542, 1934.292, 1934.042, 1933.792, 1933.542, 1933.292, 1933.042, 1932.792, 1932.542, 1932.292, 1932.042, 1931.792, 1931.542, 1931.292, 1931.042, 1930.792, 1930.542, 1930.292, 1930.042, 1929.792, 1929.542, 1929.292, 1929.042, 1928.792, 1928.542, 1928.292, 1928.042, 1927.792, 1927.542, 1927.292, 1927.042, 1926.792, 1926.542, 1926.292, 1926.042, 1925.792, 1925.542, 1925.292, 1925.042, 1924.792, 1924.542, 1924.292, 1924.042, 1923.792, 1923.542, 1923.292, 1923.042, 1922.792, 1922.542, 1922.292, 1922.042, 1921.792, 1921.542, 1921.292, 1921.042, 1920.792, 1920.542, 1920.292, 1920.042, 1919.792, 1919.542, 1919.292, 1919.042, 1918.792, 1918.542, 1918.292, 1918.042, 1917.792, 1917.542, 1917.292, 1917.042, 1916.792, 1916.542, 1916.292, 1916.042, 1915.792, 1915.542, 1915.292, 1915.042, 1914.792, 1914.542, 1914.292, 1914.042, 1913.792, 1913.542, 1913.292, 1913.042, 1912.792, 1912.542, 1912.292, 1912.042, 1911.792, 1911.542, 1911.292, 1911.042, 1910.792, 1910.542, 1910.292, 1910.042, 1909.792, 1909.542, 1909.292, 1909.042, 1908.792, 1908.542, 1908.292, 1908.042, 1907.792, 1907.542, 1907.292, 1907.042, 1906.792, 1906.542, 1906.292, 1906.042, 1905.792, 1905.542, 1905.292, 1905.042, 1904.792, 1904.542, 1904.292, 1904.042, 1903.792, 1903.542, 1903.292, 1903.042, 1902.792, 1902.542, 1902.292, 1902.042, 1901.792, 1901.542, 1901.292, 1901.042, 1900.792, 1900.542, 1900.292, 1900.042, 1899.792, 1899.542, 1899.292, 1899.042, 1898.792, 1898.542, 1898.292, 1898.042, 1897.792, 1897.542, 1897.292, 1897.042, 1896.792, 1896.542, 1896.292, 1896.042, 1895.792, 1895.542, 1895.292, 1895.042, 1894.792, 1894.542, 1894.292, 1894.042, 1893.792, 1893.542, 1893.292, 1893.042, 1892.792, 1892.542, 1892.292, 1892.042, 1891.792, 1891.542, 1891.292, 1891.042, 1890.792, 1890.542, 1890.292, 1890.042, 1889.792, 1889.542, 1889.292, 1889.042, 1888.792, 1888.542, 1888.292, 1888.042, 1887.792, 1887.542, 1887.292, 1887.042, 1886.792, 1886.542, 1886.292, 1886.042, 1885.792, 1885.542, 1885.292, 1885.042, 1884.792, 1884.542, 1884.292, 1884.042, 1883.792, 1883.542, 1883.292, 1883.042, 1882.792, 1882.542, 1882.292, 1882.042, 1881.792, 1881.542, 1881.292, 1881.042, 1880.792]}]}]}], 'googleMetadataWorksheet': 'oruuxfm', 'studyName': 'Madang, Papua New Guinea oxygen isotope record 1880-1993', 'inCompilation2_': 'PAGES2k_v2.0.0', 'googleSpreadSheetKey': '1wf30P-s54OTBdLw4dyeaIN53VDoN0u_hOqIwvAeLxtc', 'createdBy': 'matlab', 'dataContributor': {'name': 'Wu KLD'}, 'googleDataURL': 'https://docs.google.com/spreadsheets/d/1wf30P-s54OTBdLw4dyeaIN53VDoN0u_hOqIwvAeLxtc', 'minYear': 1880.792, 'datasetId': 'm8yv2VgG97zJmSg3XhqQ', 'archiveType': 'coral'}
- get_model_properties()[source]
Get all the properties associated with a model
- Returns:
A list of unique properties attached to models
- Return type:
List
Examples
from pylipd.utils.dataset import load_datasets lipd = load_datasets(names='ODP846') model_properties = lipd.get_model_properties() print(model_properties)
Loading 1 LiPD files
Loaded.. ['foundInEnsembleTable', 'method', 'type']
- get_timeseries(dsnames, to_dataframe=False)[source]
Get Legacy LiPD like Time Series Object (tso)
- Parameters:
dsnames (list) – array of dataset id or name strings
to_dataframe (bool {True; False}) – Whether to return a dataframe along the dictionary. Default is False
- Returns:
ts (dict) – A dictionary containing Time Series Object
df (Pandas.DataFrame) – If to_dataframe is set to True, returns a queriable Pandas DataFrame
Examples
from pylipd.lipd import LiPD # Fetch LiPD data from remote RDF Graph lipd_remote = LiPD() lipd_remote.set_endpoint("https://linkedearth.graphdb.mint.isi.edu/repositories/LiPDVerse2") ts_list = lipd_remote.get_timeseries(["Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001", "MD98_2181.Stott.2007", "Ant-WAIS-Divide.Severinghaus.2012"]) for dsname, tsos in ts_list.items(): for tso in tsos: if 'paleoData_variableName' in tso: print(dsname+': '+tso['paleoData_variableName']+': '+tso['archiveType'])
Extracting timeseries from dataset: Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001 ... Extracting timeseries from dataset: MD98_2181.Stott.2007 ... Extracting timeseries from dataset: Ant-WAIS-Divide.Severinghaus.2012 ...
- get_timeseries_essentials(dsname=None, mode='paleo')[source]
- Returns specific properties for timeseries: ‘dataSetName’, ‘archiveType’, ‘geo_meanLat’, ‘geo_meanLon’,
‘geo_meanElev’, ‘paleoData_variableName’, ‘paleoData_values’, ‘paleoData_units’, ‘paleoData_proxy’ (paleo only), ‘paleoData_proxyGeneral’ (paleo only), ‘time_variableName’, ‘time_values’, ‘time_units’, ‘depth_variableName’, ‘depth_values’, ‘depth_units’
- Parameters:
dsname (str, optional) – The name of the dataset for which to return the timeseries information. The default is None.
mode (paleo, chron) – Whether to retrun the information stored in the PaleoMeasurementTable or the ChronMeasurementTable. The default is ‘paleo’.
- Raises:
ValueError – Need to select either ‘chron’ or ‘paleo’
- Returns:
qres_df – A pandas dataframe returning the properties in columns for each series stored in a row of the dataframe
- Return type:
pandas.DataFrame
Example
from pylipd.utils.dataset import load_datasets lipd = load_datasets('ODP846.Lawrence.2006.lpd') df_paleo = lipd.get_timeseries_essentials(mode='paleo') print(df_paleo)
Loading 1 LiPD files
Loaded.. dataSetName archiveType geo_meanLat geo_meanLon \ 0 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 1 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 2 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 3 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 4 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 5 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 6 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 7 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 8 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 9 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 10 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 11 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 12 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 13 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 14 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 15 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 16 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 17 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 18 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 19 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 20 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 21 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 22 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 23 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 24 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 geo_meanElev paleoData_variableName \ 0 -3296.0 ukprime37.Uk37Prime 1 -3296.0 interval 2 -3296.0 sample label 3 -3296.0 sample label 4 -3296.0 sample label 5 -3296.0 c37 total.Alkenone 6 -3296.0 temp muller 7 -3296.0 temp prahl 8 -3296.0 u. peregrina d18o.d18O 9 -3296.0 u. peregrina d18o.d18O 10 -3296.0 u. peregrina d18o.d18O 11 -3296.0 c. wuellerstorfi d18o.d18O 12 -3296.0 c. wuellerstorfi d18o.d18O 13 -3296.0 c. wuellerstorfi d18o.d18O 14 -3296.0 event 15 -3296.0 event 16 -3296.0 event 17 -3296.0 site/hole 18 -3296.0 section 19 -3296.0 u. peregrina d13c.d13C 20 -3296.0 u. peregrina d13c.d13C 21 -3296.0 u. peregrina d13c.d13C 22 -3296.0 c. wuellerstorfi d13c.d13C 23 -3296.0 c. wuellerstorfi d13c.d13C 24 -3296.0 c. wuellerstorfi d13c.d13C paleoData_values paleoData_units \ 0 [0.821, 0.824, 0.828, 0.787, 0.777, 0.767, 0.7... unitless 1 [] cm 2 [] unitless 3 [] unitless 4 [] unitless 5 [2.37, 2.1, 1.87, 2.74, 3.75, 7.62, 7.86, 7.73... nmol/kg 6 [23.545, 23.648, 23.752, 22.515, 22.206, 21.89... deg C 7 [23.0, 23.1, 23.2, 22.0, 21.7, 21.4, 21.7, 21.... deg C 8 [0.14, 0.01, -0.1, -0.06, -0.17, -0.21, -0.41,... per mil PDB 9 [0.14, 0.01, -0.1, -0.06, -0.17, -0.21, -0.41,... per mil PDB 10 [0.14, 0.01, -0.1, -0.06, -0.17, -0.21, -0.41,... per mil PDB 11 [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... per mil PDB 12 [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... per mil PDB 13 [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... per mil PDB 14 [] unitless 15 [] unitless 16 [] unitless 17 [] unitless 18 [] unitless 19 [nan, nan, nan, nan, nan, nan, nan, nan, nan, ... per mil PDB 20 [nan, nan, nan, nan, nan, nan, nan, nan, nan, ... per mil PDB 21 [nan, nan, nan, nan, nan, nan, nan, nan, nan, ... per mil PDB 22 [3.38, 3.46, 3.65, 3.88, 4.14, 4.47, 4.99, 4.9... per mil PDB 23 [3.38, 3.46, 3.65, 3.88, 4.14, 4.47, 4.99, 4.9... per mil PDB 24 [3.38, 3.46, 3.65, 3.88, 4.14, 4.47, 4.99, 4.9... per mil PDB paleoData_proxy paleoData_proxyGeneral time_variableName \ 0 None None age 1 None None age 2 None None None 3 None None None 4 None None None 5 None None age 6 None None age 7 None None age 8 None None None 9 None None None 10 None None None 11 None None None 12 None None None 13 None None None 14 None None None 15 None None None 16 None None None 17 None None age 18 None None age 19 None None None 20 None None None 21 None None None 22 None None None 23 None None None 24 None None None time_values time_units \ 0 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 1 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 2 None None 3 None None 4 None None 5 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 6 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 7 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 8 None None 9 None None 10 None None 11 None None 12 None None 13 None None 14 None None 15 None None 16 None None 17 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 18 [5.228, 8.947, 11.966, 14.427, 16.502, 18.41, ... kyr BP 19 None None 20 None None 21 None None 22 None None 23 None None 24 None None depth_variableName depth_values \ 0 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 1 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 2 depth [] 3 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 4 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... 5 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 6 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 7 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 8 depth [] 9 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 10 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... 11 depth [] 12 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 13 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... 14 depth [] 15 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 16 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... 17 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 18 depth [0.16, 0.26, 0.36, 0.46, 0.56, 0.66, 0.76, 0.8... 19 depth [] 20 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 21 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... 22 depth [] 23 depth comp [12.0, 23.0, 33.0, 33.0, 43.0, 53.0, 63.0, 73.... 24 depth cr [0.12, 0.23, 0.33, 0.33, 0.43, 0.53, 0.63, 0.7... depth_units 0 m 1 m 2 m 3 mcd 4 rmcd 5 m 6 m 7 m 8 m 9 mcd 10 rmcd 11 m 12 mcd 13 rmcd 14 m 15 mcd 16 rmcd 17 m 18 m 19 m 20 mcd 21 rmcd 22 m 23 mcd 24 rmcdTo return the information stored in the ChronTable:
from pylipd.utils.dataset import load_datasets lipd = load_datasets('ODP846.Lawrence.2006.lpd') df_chron = lipd.get_timeseries_essentials(mode='chron') print(df_chron)
Loading 1 LiPD files
Loaded.. dataSetName archiveType geo_meanLat geo_meanLon \ 0 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 1 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 2 ODP846.Lawrence.2006 marine sediment -3.1 -90.8 geo_meanElev chronData_variableName \ 0 -3296.0 age 1 -3296.0 depth 2 -3296.0 d18o chronData_values chronData_units \ 0 [3.645, 7.99, 11.18, 13.803, 15.886, 17.93, 19... ky BP 1 [0.12, 0.23, 0.33, 0.43, 0.53, 0.63, 0.73, 0.8... m 2 [3.38, 3.46, 3.765, 4.14, 4.47, 4.99, 4.99, 4.... per mil time_variableName time_values \ 0 age [3.645, 7.99, 11.18, 13.803, 15.886, 17.93, 19... 1 age [3.645, 7.99, 11.18, 13.803, 15.886, 17.93, 19... 2 age [3.645, 7.99, 11.18, 13.803, 15.886, 17.93, 19... time_units depth_variableName \ 0 ky BP depth 1 ky BP depth 2 ky BP depth depth_values depth_units 0 [0.12, 0.23, 0.33, 0.43, 0.53, 0.63, 0.73, 0.8... m 1 [0.12, 0.23, 0.33, 0.43, 0.53, 0.63, 0.73, 0.8... m 2 [0.12, 0.23, 0.33, 0.43, 0.53, 0.63, 0.73, 0.8... m
- get_variable_properties()[source]
Get a list of variable properties that can be used for querying
- Returns:
A list of unique variable properties
- Return type:
list
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir(name='Pages2k') variable_properties = lipd.get_variable_properties() print(variable_properties)
Loading 16 LiPD files
Loaded.. ['foundInTable', 'useInGlobalTemperatureAnalysis', 'name', 'paleoDataTableName', 'inferredVariableType', 'partOfCompilation', 'hasMinValue', 'hasColumnNumber', 'qCnotes', 'hasMaxValue', 'proxy', 'calibratedVia', 'hasMeanValue', 'wDSPaleoUrl', 'foundInDataset', 'pages2kID', 'type', 'hasResolution', 'qCCertification', 'hasVariableID', 'measurementTableName', 'archiveType', 'hasUnits', 'hasValues', 'hasMedianValue', 'measurementTableMD5', 'interpretedAs', 'dataType', 'description', 'proxyObservationType', 'hasProxySystem', 'notes', 'detail', 'sensorGenus', 'sensorSpecies', 'ocean2kID', 'iso2kUI', 'hasUncertainty', 'method', 'measurementMaterial', 'precededBy']
- load(lipdfiles, parallel=False)[source]
Load LiPD files.
- Parameters:
lipdfiles (list of str) – array of paths to lipd files (the paths could also be urls)
parallel (bool) – (Optional) set to True to process lipd files in parallel. You must run this function under the “__main__” process for this to work
Examples
In this example, we load LiPD files for an array of paths.
from pylipd.lipd import LiPD lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd", "../examples/data/Ant-WAIS-Divide.Severinghaus.2012.lpd", "https://lipdverse.org/data/LCf20b99dfe8d78840ca60dfb1f832b9ec/1_0_1/Nunalleq.Ledger.2018.lpd" ]) print(lipd.get_all_dataset_names())
Loading 4 LiPD files
Loaded.. ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007', 'Ant-WAIS-Divide.Severinghaus.2012', 'Nunalleq.Ledger.2018']
- load_from_dir(dir_path, parallel=False, cutoff=None)[source]
Load LiPD files from a directory
- Parameters:
dir_path (str) – path to the directory containing lipd files
parallel (bool) – (Optional) set to True to process lipd files in parallel. You must run this function under the “__main__” process for this to work
cutoff (int) – (Optional) the maximum number of files to load at once.
Examples
In this example, we load LiPD files from a directory.
from pylipd.lipd import LiPD lipd = LiPD() lipd.load_from_dir("../examples/data") print(lipd.get_all_dataset_names())
Loading 4 LiPD files
Loaded.. ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007', 'Ant-WAIS-Divide.Severinghaus.2012', 'ODP846.Lawrence.2006']
- load_remote_datasets(dsnames)[source]
Loads remote datasets into cache if a remote endpoint is set
- Parameters:
dsnames (array) – array of dataset names
Examples
from pylipd.lipd import LiPD # Fetch LiPD data from remote RDF Graph lipd_remote = LiPD() lipd_remote.set_endpoint("https://linkedearth.graphdb.mint.isi.edu/repositories/LiPDVerse2") lipd_remote.load_remote_datasets(["Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001", "MD98_2181.Stott.2007", "Ant-WAIS-Divide.Severinghaus.2012"]) print(lipd_remote.get_all_dataset_names())
Caching datasets from remote endpoint.. Making remote query to endpoint: https://linkedearth.graphdb.mint.isi.edu/repositories/LiPDVerse2
Done.. ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007', 'Ant-WAIS-Divide.Severinghaus.2012']
- pop(dsnames)[source]
Pops dataset(s) from the graph and returns the popped LiPD object
- Parameters:
dsnames (str or list of str) – dataset name(s) to be popped.
- Returns:
LiPD object with the popped dataset(s)
- Return type:
Examples
from pylipd.lipd import LiPD # Load local files lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) all_datasets = lipd.get_all_dataset_names() print("Loaded datasets: " + str(all_datasets)) popped = lipd.pop(all_datasets[0]) print("Loaded datasets after pop: " + str(lipd.get_all_dataset_names())) print("Popped dataset: " + str(popped.get_all_dataset_names()))
Loading 2 LiPD files
Loaded.. Loaded datasets: ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007'] Loaded datasets after pop: ['MD98_2181.Stott.2007'] Popped dataset: ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001']
- remove(dsnames)[source]
Removes dataset(s) from the graph
- Parameters:
dsnames (str or list of str) – dataset name(s) to be removed
Examples
from pylipd.lipd import LiPD # Load local files lipd = LiPD() lipd.load([ "../examples/data/Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001.lpd", "../examples/data/MD98_2181.Stott.2007.lpd" ]) all_datasets = lipd.get_all_dataset_names() print("Loaded datasets: " + str(all_datasets)) lipd.remove(all_datasets[0]) print("Loaded datasets after remove: " + str(lipd.get_all_dataset_names()))
Loading 2 LiPD files
Loaded.. Loaded datasets: ['Ocn-MadangLagoonPapuaNewGuinea.Kuhnert.2001', 'MD98_2181.Stott.2007'] Loaded datasets after remove: ['MD98_2181.Stott.2007']
- to_lipd_series(parallel=False)[source]
Converts the LiPD object to a LiPDSeries object
- Parameters:
parallel (bool) – Whether to use parallel processing to load the data. Default is False
- Returns:
A LiPDSeries object
- Return type:
pylipd.lipd.LiPDSeries
Examples
from pylipd.lipd import LiPD lipd = LiPD() lipd.load([ "../examples/data/ODP846.Lawrence.2006.lpd" ]) S = lipd.to_lipd_series()
Loading 1 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done..
LiPDSeries (pylipd.lipd_series.LiPDSeries)
- class pylipd.lipd_series.LiPDSeries(graph=None)[source]
The LiPD Series class describes a collection of LiPD (Linked Paleo Data) variables. It contains an RDF Graph which is serialization of LiPD variables into an RDF graph containing terms from the LiPD Ontology <http://linked.earth/Ontology/release/core/1.2.0/index-en.html>. Each LiPD Variable is also associated with the LiPD itself so it can be deserialized into the original LiPD format. How to browse and query the LiPD variables is described in a short example below.
Examples
In this example, we read an online LiPD file and convert it into a time series object dictionary.
from pylipd.lipd_series import LiPDSeries lipd = LiPD() lipd.load(["https://lipdverse.org/data/LCf20b99dfe8d78840ca60dfb1f832b9ec/1_0_1//Nunalleq.Ledger.2018.lpd"]) lipd_series = lipd.to_lipd_series()
Loading 1 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done..
Methods
clear()Clears the graph
copy()Makes a copy of the object
filter_by_name(name)Filters series to return a new LiPDSeries that only keeps variables that have the specified name (regex)
get(ids)Get id(s) from the graph and returns the LiPD object
get_all_graph_ids()Get all Graph ids
Get a list of all possible distinct variableNames.
Returns a list of all variables in the graph
This function returns information about each variable: dataSetName, archiveType, name, values, units, TSID, proxy.
load(lipd[, parallel])Extract Variables from the LiPD object.
merge(rdf)Merges the current LiPD object with another LiPD object
pop(ids)Pops graph(s) from the combined graph and returns the popped RDF Graph
query(query[, remote, result])Once data is loaded into the graph (or remote endpoint set), one can make SparQL queries to the graph
remove(ids)Removes ids(s) from the graph
serialize()Returns RDF quad serialization of the current combined Graph .
set_endpoint(endpoint)Sets a SparQL endpoint for a remote Knowledge Base (example: GraphDB)
- filter_by_name(name)[source]
Filters series to return a new LiPDSeries that only keeps variables that have the specified name (regex)
- Parameters:
name (str) – The variable name to filter by
- Returns:
A new LiPDSeries object that only contains variables that have the specified name (regex)
- Return type:
- get_all_variable_names()[source]
Get a list of all possible distinct variableNames. Useful for filtering and qeurying.
- Returns:
A list of unique variableName
- Return type:
list
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir('Pages2k') S = lipd.to_lipd_series() varName = S.get_all_variable_names() print(varName)
Loading 16 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done.. ['year', 'temperature', 'uncertainty_temperature', 'd18O', 'MXD', 'Mg_Ca.Mg/Ca', 'depth_top', 'notes', 'depth_bottom', 'trsgi', 'Uk37']
- get_all_variables()[source]
Returns a list of all variables in the graph
- Returns:
A dataframe of all variables in the graph with columns uri, varid, varname
- Return type:
pandas.DataFrame
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir() S = lipd.to_lipd_series() df = S.get_all_variables() print(df)
Loading 16 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done.. uri TSID \ 0 http://linked.earth/lipd/Ant-WAIS-Divide.Sever... PYTAAFWZCUK 1 http://linked.earth/lipd/Ant-WAIS-Divide.Sever... LPDb9285123 2 http://linked.earth/lipd/Ant-WAIS-Divide.Sever... LPDa7a4074f 3 http://linked.earth/lipd/Ocn-PedradeLume-CapeV... Ocean2kHR_107 4 http://linked.earth/lipd/Ocn-PedradeLume-CapeV... PYT296KN772 5 http://linked.earth/lipd/Ocn-AlboranSea436B.Ni... LPD0e0867fe 6 http://linked.earth/lipd/Ocn-AlboranSea436B.Ni... PYTPD2RJATT 7 http://linked.earth/lipd/Eur-NorthernScandinav... PYTECO66XAD 8 http://linked.earth/lipd/Eur-NorthernScandinav... Eur_014 9 http://linked.earth/lipd/Eur-CoastofPortugal.A... Eur_010 10 http://linked.earth/lipd/Eur-CoastofPortugal.A... PYTNSKF0DVD 11 http://linked.earth/lipd/Eur-NorthernSpain.Mar... Eur_008 12 http://linked.earth/lipd/Eur-NorthernSpain.Mar... PYTE7VH7UMO 13 http://linked.earth/lipd/Ocn-SinaiPeninsula_Re... PYTGGLQ9T54 14 http://linked.earth/lipd/Ocn-SinaiPeninsula_Re... Ocean2kHR_018 15 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPDba471ae7 16 http://linked.earth/lipd/Ocn-FeniDrift.Richter... PYTHS7WC58V 17 http://linked.earth/lipd/Ocn-FeniDrift.Richter... PYTA482M43E 18 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPD873b43d0 19 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPD6e0eacd1 20 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPDbceb5d84 21 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPDb7290ea4 22 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPD47182517 23 http://linked.earth/lipd/Ocn-FeniDrift.Richter... LPD071f9a58 24 http://linked.earth/lipd/Asi-SourthAndMiddleUr... PYTX5TD5SOT 25 http://linked.earth/lipd/Asi-SourthAndMiddleUr... Asia_230 26 http://linked.earth/lipd/Eur-SpannagelCave.Man... PYTSOOGT8HT 27 http://linked.earth/lipd/Eur-SpannagelCave.Man... Eur_001 28 http://linked.earth/lipd/Ocn-RedSea.Felis.2000... Ocean2kHR_019 29 http://linked.earth/lipd/Ocn-RedSea.Felis.2000... PYTXPC7HUA2 30 http://linked.earth/lipd/Eur-SpanishPyrenees.D... PYT2K8MIA3N 31 http://linked.earth/lipd/Eur-SpanishPyrenees.D... Eur_020 32 http://linked.earth/lipd/Eur-Stockholm.Leijonh... PYTWVH672OU 33 http://linked.earth/lipd/Eur-Stockholm.Leijonh... Eur_006 34 http://linked.earth/lipd/Eur-FinnishLakelands.... Eur_005 35 http://linked.earth/lipd/Eur-FinnishLakelands.... PYTUSB62S0A 36 http://linked.earth/lipd/Eur-LakeSilvaplana.Tr... Eur_002 37 http://linked.earth/lipd/Eur-LakeSilvaplana.Tr... PYT1E4X3DDF 38 http://linked.earth/lipd/Arc-Kongressvatnet.D_... Arc_077 39 http://linked.earth/lipd/Arc-Kongressvatnet.D_... Arc_078 40 http://linked.earth/lipd/Arc-Kongressvatnet.D_... PYTOAVDFCGU variableName 0 year 1 temperature 2 uncertainty_temperature 3 d18O 4 year 5 temperature 6 year 7 year 8 MXD 9 temperature 10 year 11 d18O 12 year 13 year 14 d18O 15 Mg_Ca.Mg/Ca 16 year 17 year 18 depth_top 19 notes 20 Mg_Ca.Mg/Ca 21 temperature 22 temperature 23 depth_bottom 24 year 25 temperature 26 year 27 d18O 28 d18O 29 year 30 year 31 trsgi 32 year 33 temperature 34 temperature 35 year 36 temperature 37 year 38 Uk37 39 temperature 40 year
- get_timeseries_essentials()[source]
This function returns information about each variable: dataSetName, archiveType, name, values, units, TSID, proxy.
- Returns:
qres_df – A dataframe containing the information in each column
- Return type:
pandas.DataFrame
Examples
from pylipd.utils.dataset import load_dir lipd = load_dir() S = lipd.to_lipd_series() df = S.get_timeseries_essentials() print(df)
Loading 16 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done.. dataSetName archiveType \ 0 Ant-WAIS-Divide.Severinghaus.2012 borehole 1 Ant-WAIS-Divide.Severinghaus.2012 borehole 2 Ant-WAIS-Divide.Severinghaus.2012 borehole 3 Ocn-PedradeLume-CapeVerdeIslands.Moses.2006 coral 4 Ocn-PedradeLume-CapeVerdeIslands.Moses.2006 coral 5 Ocn-AlboranSea436B.Nieto-Moreno.2013 marine sediment 6 Ocn-AlboranSea436B.Nieto-Moreno.2013 marine sediment 7 Eur-NorthernScandinavia.Esper.2012 tree 8 Eur-NorthernScandinavia.Esper.2012 tree 9 Eur-CoastofPortugal.Abrantes.2011 marine sediment 10 Eur-CoastofPortugal.Abrantes.2011 marine sediment 11 Eur-NorthernSpain.Martin-Chivelet.2011 speleothem 12 Eur-NorthernSpain.Martin-Chivelet.2011 speleothem 13 Ocn-SinaiPeninsula,RedSea.Moustafa.2000 coral 14 Ocn-SinaiPeninsula,RedSea.Moustafa.2000 coral 15 Ocn-FeniDrift.Richter.2009 marine sediment 16 Ocn-FeniDrift.Richter.2009 marine sediment 17 Ocn-FeniDrift.Richter.2009 marine sediment 18 Ocn-FeniDrift.Richter.2009 marine sediment 19 Ocn-FeniDrift.Richter.2009 marine sediment 20 Ocn-FeniDrift.Richter.2009 marine sediment 21 Ocn-FeniDrift.Richter.2009 marine sediment 22 Ocn-FeniDrift.Richter.2009 marine sediment 23 Ocn-FeniDrift.Richter.2009 marine sediment 24 Asi-SourthAndMiddleUrals.Demezhko.2007 borehole 25 Asi-SourthAndMiddleUrals.Demezhko.2007 borehole 26 Eur-SpannagelCave.Mangini.2005 speleothem 27 Eur-SpannagelCave.Mangini.2005 speleothem 28 Ocn-RedSea.Felis.2000 coral 29 Ocn-RedSea.Felis.2000 coral 30 Eur-SpanishPyrenees.Dorado-Linan.2012 tree 31 Eur-SpanishPyrenees.Dorado-Linan.2012 tree 32 Eur-Stockholm.Leijonhufvud.2009 documents 33 Eur-Stockholm.Leijonhufvud.2009 documents 34 Eur-FinnishLakelands.Helama.2014 tree 35 Eur-FinnishLakelands.Helama.2014 tree 36 Eur-LakeSilvaplana.Trachsel.2010 lake sediment 37 Eur-LakeSilvaplana.Trachsel.2010 lake sediment 38 Arc-Kongressvatnet.D'Andrea.2012 lake sediment 39 Arc-Kongressvatnet.D'Andrea.2012 lake sediment 40 Arc-Kongressvatnet.D'Andrea.2012 lake sediment name TSID \ 0 year PYTAAFWZCUK 1 temperature LPDb9285123 2 uncertainty_temperature LPDa7a4074f 3 year PYT296KN772 4 d18O Ocean2kHR_107 5 year PYTPD2RJATT 6 temperature LPD0e0867fe 7 year PYTECO66XAD 8 MXD Eur_014 9 year PYTNSKF0DVD 10 temperature Eur_010 11 d18O Eur_008 12 year PYTE7VH7UMO 13 d18O Ocean2kHR_018 14 year PYTGGLQ9T54 15 temperature LPDb7290ea4 16 depth_top LPD873b43d0 17 notes LPD6e0eacd1 18 depth_bottom LPD071f9a58 19 temperature LPD47182517 20 Mg_Ca.Mg/Ca LPDbceb5d84 21 Mg_Ca.Mg/Ca LPDba471ae7 22 year PYTA482M43E 23 year PYTHS7WC58V 24 temperature Asia_230 25 year PYTX5TD5SOT 26 d18O Eur_001 27 year PYTSOOGT8HT 28 d18O Ocean2kHR_019 29 year PYTXPC7HUA2 30 trsgi Eur_020 31 year PYT2K8MIA3N 32 temperature Eur_006 33 year PYTWVH672OU 34 year PYTUSB62S0A 35 temperature Eur_005 36 temperature Eur_002 37 year PYT1E4X3DDF 38 Uk37 Arc_077 39 temperature Arc_078 40 year PYTOAVDFCGU values units proxy 0 [8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0,... AD None 1 [-29.607, -29.607, -29.606, -29.606, -29.605, ... degC borehole 2 [1.327, 1.328, 1.328, 1.329, 1.33, 1.33, 1.331... degC None 3 [1928.96, 1929.04, 1929.12, 1929.21, 1929.29, ... AD None 4 [-3.11, -2.9, -2.88, -2.73, -2.73, -2.84, -2.8... permil d18O 5 [1999.07, 1993.12, 1987.17, 1975.26, 1963.36, ... AD None 6 [18.79, 19.38, 19.61, 18.88, 18.74, 19.25, 18.... degC alkenone 7 [-138.0, -137.0, -136.0, -135.0, -134.0, -133.... AD None 8 [0.46, 1.305, 0.755, -0.1, -0.457, 1.62, 0.765... None MXD 9 [971.19, 982.672, 991.858, 1001.044, 1010.23, ... AD None 10 [15.235, 15.329, 15.264, 15.376, 15.4, 15.129,... degC alkenone 11 [0.94, 0.8, 0.23, 0.17, 0.51, 0.36, 0.24, 0.4,... permil d18O 12 [2000.0, 1987.0, 1983.0, 1978.0, 1975.0, 1971.... AD None 13 [-3.05, -3.63, -3.53, -3.47, -3.1, -3.45, -3.6... permil d18O 14 [1993.12, 1992.86, 1992.66, 1992.39, 1992.12, ... AD None 15 [12.94, 10.99, 10.53, 10.44, 11.39, 13.38, 10.... degC None 16 [0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, ... cm None 17 [] None None 18 [0.5, 1.5, 2.5, 3.5, 4.5, 5.5, 6.5, 7.5, 8.5, ... cm None 19 [12.94, 10.99, 10.53, 10.44, 11.39, 13.38, 10.... degC foram Mg/Ca 20 [2.31, 1.973, 1.901, 1.887, 2.038, 2.394, 1.83... None None 21 [2.31, 1.973, 1.901, 1.887, 2.038, 2.394, 1.83... None None 22 [1998.0, 1987.0, 1975.0, 1962.0, 1949.0, 1936.... AD None 23 [1998.0, 1987.0, 1975.0, 1962.0, 1949.0, 1936.... AD None 24 [0.166, 0.264, 0.354, 0.447, 0.538, 0.62, 0.68... degC borehole 25 [800.0, 850.0, 900.0, 950.0, 1000.0, 1050.0, 1... AD None 26 [-7.49, -7.41, -7.36, -7.15, -7.28, -6.99, -6.... permil d18O 27 [1935.0, 1932.0, 1930.0, 1929.0, 1929.0, 1928.... AD None 28 [-4.12, -3.82, -3.05, -3.02, -3.62, -3.96, -3.... permil d18O 29 [1995.583, 1995.417, 1995.25, 1995.083, 1994.9... AD None 30 [-1.612, -0.703, -0.36, -0.767, -0.601, -0.733... None TRW 31 [1260.0, 1261.0, 1262.0, 1263.0, 1264.0, 1265.... AD None 32 [-1.7212, -1.6382, -0.6422, 0.1048, -0.7252, -... degC historic 33 [1502.0, 1503.0, 1504.0, 1505.0, 1506.0, 1507.... AD None 34 [2000.0, 1999.0, 1998.0, 1997.0, 1996.0, 1995.... AD None 35 [14.603, 14.643, 12.074, 13.898, 13.671, 13.41... degC MXD 36 [0.181707222, 0.111082797, 0.001382129, -0.008... degC reflectance 37 [1175.0, 1176.0, 1177.0, 1178.0, 1179.0, 1180.... AD None 38 [-0.65, -0.67, -0.65, -0.67, -0.69, -0.68, -0.... None None 39 [5.9, 5.1, 6.1, 5.3, 4.3, 4.8, 3.8, 4.8, 4.3, ... degC alkenone 40 [2008.0, 2004.0, 2000.0, 1996.0, 1990.0, 1987.... AD None
- load(lipd, parallel=False)[source]
Extract Variables from the LiPD object.
- Parameters:
lipd (LiPD) – A LiPD object
Examples
from pylipd.lipd_series import LiPDSeries lipd = LiPD() lipd.load(["https://lipdverse.org/data/LCf20b99dfe8d78840ca60dfb1f832b9ec/1_0_1//Nunalleq.Ledger.2018.lpd"]) lipd_series = lipd.to_lipd_series()
Loading 1 LiPD files
Loaded.. Creating LiPD Series... - Extracting dataset subgraphs
- Extracting variable subgraphs
Done..