How to use the generator utility

This tool allows to generate synthetic photometry in a set of available systems from the input internally-calibrated continuously-represented mean spectra.

Information about the units of the variables used in this tutorial can be found here.

# Import the tool and the PhotometricSystem class
from gaiaxpy import generate, PhotometricSystem

We can see which systems are available.

PhotometricSystem.get_available_systems()

'Els_Custom_W09_S2, Euclid_VIS, Gaia_2, Gaia_DR3_Vega, Halpha_Custom_AB, H_Custom, Hipparcos_Tycho, HST_ACSWFC, HST_HUGS_Std, HST_WFC3UVIS, HST_WFPC2, IPHAS, JKC, JKC_Std, JPAS, JPLUS, JWST_NIRCAM, PanSTARRS1, PanSTARRS1_Std, Pristine, SDSS, SDSS_Std, Stromgren, Stromgren_Std, WFIRST'

For a number of systems a standardisation procedure has been carried out to minimise zero point differences and/or trends as a function of colour (see Montegriffo et al. 2022). These are available in GaiaXPy by using the "_Std" version of the desired photometric system.

Basic usage

The generator returns a single output:

1. A DataFrame of all synthetic photometry results. The DataFrame contains the magnitudes, fluxes and flux errors for each band in the system.

# Path to file with XP CONTINUOUS RAW data (csv, ecsv, fits, or xml)
f = '/path/to/XP_CONTINUOUS_RAW.xml'
# Select a photometric system
phot_system = PhotometricSystem.Gaia_DR3_Vega
synthetic_photometry = generate(f, photometric_system=phot_system)
synthetic_photometry

	source_id	GaiaDr3Vega_mag_G	GaiaDr3Vega_mag_BP	GaiaDr3Vega_mag_RP	GaiaDr3Vega_mag_RVS_RP	GaiaDr3Vega_mag_RVS	GaiaDr3Vega_flux_G	GaiaDr3Vega_flux_BP	GaiaDr3Vega_flux_RP	GaiaDr3Vega_flux_RVS_RP	GaiaDr3Vega_flux_RVS	GaiaDr3Vega_flux_error_G	GaiaDr3Vega_flux_error_BP	GaiaDr3Vega_flux_error_RP	GaiaDr3Vega_flux_error_RVS_RP	GaiaDr3Vega_flux_error_RVS
0	6	8.953604	11.353152	7.562385	6.961298	6.956901	6.646585e-15	1.181787e-15	1.225988e-14	1.497480e-14	1.502310e-14	2.687982e-18	1.330194e-18	4.299604e-18	3.048522e-17	3.056536e-17
1	4	13.172242	13.056574	13.400146	13.455164	13.455571	1.365033e-16	2.461334e-16	5.667372e-17	3.782810e-17	3.778257e-17	5.450830e-20	1.521506e-19	2.870977e-20	1.607094e-19	1.603705e-19

A list of PhotometricSystem objects is also accepted.

# Create a list of photometric systems
phot_system_list = [PhotometricSystem.Gaia_DR3_Vega, PhotometricSystem.JKC, PhotometricSystem.Pristine]
synthetic_photometry = generate(f, photometric_system=phot_system_list)
synthetic_photometry

	source_id	GaiaDr3Vega_mag_G	GaiaDr3Vega_mag_BP	GaiaDr3Vega_mag_RP	GaiaDr3Vega_mag_RVS_RP	GaiaDr3Vega_mag_RVS	GaiaDr3Vega_flux_G	GaiaDr3Vega_flux_BP	GaiaDr3Vega_flux_RP	GaiaDr3Vega_flux_RVS_RP	...	Jkc_flux_R	Jkc_flux_I	Jkc_flux_error_U	Jkc_flux_error_B	Jkc_flux_error_V	Jkc_flux_error_R	Jkc_flux_error_I	Pristine_mag_CaHK	Pristine_flux_CaHK	Pristine_flux_error_CaHK
0	6	8.953604	11.353152	7.562385	6.961298	6.956901	6.646585e-15	1.181787e-15	1.225988e-14	1.497480e-14	...	3.849211e-15	1.305855e-14	5.966184e-18	7.583555e-19	1.468439e-18	2.945754e-18	7.269074e-18	13.818329	2.162462e-16	9.483388e-18
1	4	13.172242	13.056574	13.400146	13.455164	13.455571	1.365033e-16	2.461334e-16	5.667372e-17	3.782810e-17	...	1.062562e-16	5.024040e-17	2.702577e-18	3.499559e-19	1.437515e-19	8.295570e-20	3.187286e-20	13.016468	4.525774e-16	4.957204e-18

2 rows × 34 columns

Input types

The input does not have to be a path to a file. There are other options: a pandas DataFrame, an ADQL query or a list of sourceIds.

Passing a DataFrame

import pandas as pd

f = '/path/to/XP_CONTINUOUS_RAW.csv'
df = pd.read_csv(f) # The values in the DataFrame can be edited if the user wishes to do so.

synthetic_photometry = generate(df, photometric_system=PhotometricSystem.Els_Custom_W09_S2)
synthetic_photometry

	source_id	ElsCustomW09S2_mag_Halpha	ElsCustomW09S2_mag_Hbeta	ElsCustomW09S2_mag_O3	ElsCustomW09S2_mag_CHalpha	ElsCustomW09S2_mag_CHbeta	ElsCustomW09S2_mag_CO3	ElsCustomW09S2_mag_r	ElsCustomW09S2_mag_i	ElsCustomW09S2_flux_Halpha	...	ElsCustomW09S2_flux_r	ElsCustomW09S2_flux_i	ElsCustomW09S2_flux_error_Halpha	ElsCustomW09S2_flux_error_Hbeta	ElsCustomW09S2_flux_error_O3	ElsCustomW09S2_flux_error_CHalpha	ElsCustomW09S2_flux_error_CHbeta	ElsCustomW09S2_flux_error_CO3	ElsCustomW09S2_flux_error_r	ElsCustomW09S2_flux_error_i
0	6	9.322991	12.011327	11.772323	9.996499	12.418602	11.171822	10.234851	7.648620	3.623126e-15	...	2.011775e-15	1.121584e-14	1.406098e-17	3.498787e-18	3.658348e-18	1.145484e-17	4.089499e-18	3.986050e-18	2.963247e-18	7.769417e-18
1	4	13.351317	13.148944	13.131454	13.267574	13.018520	13.134297	13.261007	13.393298	8.866521e-17	...	1.239131e-16	5.648860e-17	2.775161e-19	6.395538e-19	5.618435e-19	4.977891e-19	1.078377e-18	4.220456e-19	1.054331e-19	3.647871e-20

2 rows × 25 columns

Running a query

The generator can also take the parameters 'username' and 'password' for Cosmos credentials, instead of using the interactive login (like in the example below). See that information here.

query_input = "select TOP 2 source_id from gaiadr3.gaia_source where has_xp_continuous = 'True'"
phot_system_list = [PhotometricSystem.Stromgren, PhotometricSystem.Stromgren_Std]
synthetic_photometry = generate(query_input, photometric_system=phot_system_list)
synthetic_photometry

INFO: Login to gaia TAP server [astroquery.gaia.core]
User: user
Password: ········
OK
INFO: Login to gaia data server [astroquery.gaia.core]
OK
INFO: Query finished. [astroquery.utils.tap.core]

	source_id	Stromgren_mag_u	Stromgren_mag_v	Stromgren_mag_b	Stromgren_mag_y	Stromgren_flux_u	Stromgren_flux_v	Stromgren_flux_b	Stromgren_flux_y	Stromgren_flux_error_u	...	StromgrenStd_mag_b	StromgrenStd_mag_y	StromgrenStd_flux_u	StromgrenStd_flux_v	StromgrenStd_flux_b	StromgrenStd_flux_y	StromgrenStd_flux_error_u	StromgrenStd_flux_error_v	StromgrenStd_flux_error_b	StromgrenStd_flux_error_y
0	48	17.800398	18.272564	16.719616	15.640904	2.753218e-30	1.782274e-30	7.449957e-30	2.012049e-29	2.920922e-31	...	16.901830	15.673323	2.752540e-30	1.958493e-30	7.194956e-30	1.987151e-29	2.920292e-31	1.043331e-31	9.554138e-32	1.968858e-31
1	44	17.481469	19.351867	18.399179	17.897489	3.693281e-30	6.595584e-31	1.586092e-30	2.517702e-30	5.435962e-31	...	18.599831	17.953763	3.692461e-30	6.827172e-31	1.506009e-30	2.432511e-30	5.434962e-31	6.971673e-32	5.017006e-32	7.751200e-32

2 rows × 25 columns

Passing a list

A list of sourceIds can be passed to the generator as the first argument. The generator will then query the Archive for these objects.

The generator can also take the parameters 'username' and 'password' for Cosmos credentials, instead of using the interactive login (like in the example below). See that information here.

sources_list = ['48', 44] # The sourceIds can be string or long.
synthetic_photometry = generate(sources_list, photometric_system=PhotometricSystem.JKC_Std)
synthetic_photometry

INFO: Login to gaia TAP server [astroquery.gaia.core]
User: user
Password: ········
OK
INFO: Login to gaia data server [astroquery.gaia.core]
OK

	source_id	JkcStd_mag_U	JkcStd_mag_B	JkcStd_mag_V	JkcStd_mag_R	JkcStd_mag_I	JkcStd_flux_U	JkcStd_flux_B	JkcStd_flux_V	JkcStd_flux_R	JkcStd_flux_I	JkcStd_flux_error_U	JkcStd_flux_error_B	JkcStd_flux_error_V	JkcStd_flux_error_R	JkcStd_flux_error_I
0	48	17.313257	17.476459	15.704760	14.528387	13.199215	5.434222e-18	6.622414e-18	1.921569e-17	3.450842e-17	6.192043e-17	4.475145e-19	4.778960e-20	3.725142e-20	3.929187e-20	3.980868e-20
1	44	17.333973	19.025791	17.914239	17.133965	16.512081	5.331519e-18	1.589584e-18	2.511103e-18	3.131078e-18	2.928785e-18	7.467517e-19	2.767727e-20	1.521533e-20	1.644628e-20	1.049679e-20

Advanced usage

The generator offers access to another functionality via the optional parameter "error_correction".

Error correction

An analysis based on a dataset of sources for which the available observations per source were randomly split in two sets thus leading to the generation of two mean spectra for each object, has shown that errors on synthetic photometry are somewhat underestimated.

This same dataset has been used to compute empirical corrections to the uncertainties (see Montegriffo et al. 2022). These corrections are available in GaiaXPy and can be activated when generating synthethic photometry.

# To apply the error correction, first create a list of systems.
phot_system_list = [PhotometricSystem.Euclid_VIS, PhotometricSystem.Els_Custom_W09_S2]
synthetic_phot_corrected = generate(df, photometric_system=phot_system_list, error_correction=True)
synthetic_phot_corrected
# A warning is raised for each of the systems that cannot be corrected. No changes are made to these systems.

UserWarning: System ElsCustomW09S2 does not have a correction table. The program will not apply error correction over this system.

	source_id	EuclidVis_mag_VIS	EuclidVis_flux_VIS	EuclidVis_flux_error_VIS	ElsCustomW09S2_mag_Halpha	ElsCustomW09S2_mag_Hbeta	ElsCustomW09S2_mag_O3	ElsCustomW09S2_mag_CHalpha	ElsCustomW09S2_mag_CHbeta	ElsCustomW09S2_mag_CO3	...	ElsCustomW09S2_flux_r	ElsCustomW09S2_flux_i	ElsCustomW09S2_flux_error_Halpha	ElsCustomW09S2_flux_error_Hbeta	ElsCustomW09S2_flux_error_O3	ElsCustomW09S2_flux_error_CHalpha	ElsCustomW09S2_flux_error_CHbeta	ElsCustomW09S2_flux_error_CO3	ElsCustomW09S2_flux_error_r	ElsCustomW09S2_flux_error_i
0	6	8.229913	8.775556e-15	1.483339e-17	9.322991	12.011327	11.772323	9.996499	12.418602	11.171822	...	2.011775e-15	1.121584e-14	1.406098e-17	3.498787e-18	3.658348e-18	1.145484e-17	4.089499e-18	3.986050e-18	2.963247e-18	7.769417e-18
1	4	13.318073	8.091163e-17	1.059743e-19	13.351317	13.148944	13.131454	13.267574	13.018520	13.134297	...	1.239131e-16	5.648860e-17	2.775161e-19	6.395538e-19	5.618435e-19	4.977891e-19	1.078377e-18	4.220456e-19	1.054331e-19	3.647871e-20

2 rows × 28 columns

Colour equation and error correction

Both options colour_equation and error_correction can be True at the same time. The error correction needs the Gaia_DR3_Vega system to be present in the input data, so the system is internally added and then removed before returning the results.

phot_system_list = [PhotometricSystem.Euclid_VIS, PhotometricSystem.Els_Custom_W09_S2]
synthetic_phot_corrected = generate(df, photometric_system=phot_system_list, error_correction=True)
synthetic_phot_corrected
# A warning is raised for each of the systems that cannot be corrected. No changes are made to these systems.

UserWarning: System ElsCustomW09S2 does not have a correction table. The program will not apply error correction over this system.

	source_id	EuclidVis_mag_VIS	EuclidVis_flux_VIS	EuclidVis_flux_error_VIS	ElsCustomW09S2_mag_Halpha	ElsCustomW09S2_mag_Hbeta	ElsCustomW09S2_mag_O3	ElsCustomW09S2_mag_CHalpha	ElsCustomW09S2_mag_CHbeta	ElsCustomW09S2_mag_CO3	...	ElsCustomW09S2_flux_r	ElsCustomW09S2_flux_i	ElsCustomW09S2_flux_error_Halpha	ElsCustomW09S2_flux_error_Hbeta	ElsCustomW09S2_flux_error_O3	ElsCustomW09S2_flux_error_CHalpha	ElsCustomW09S2_flux_error_CHbeta	ElsCustomW09S2_flux_error_CO3	ElsCustomW09S2_flux_error_r	ElsCustomW09S2_flux_error_i
0	6	8.229913	8.775556e-15	1.483339e-17	9.322991	12.011327	11.772323	9.996499	12.418602	11.171822	...	2.011775e-15	1.121584e-14	1.406098e-17	3.498787e-18	3.658348e-18	1.145484e-17	4.089499e-18	3.986050e-18	2.963247e-18	7.769417e-18
1	4	13.318073	8.091163e-17	1.059743e-19	13.351317	13.148944	13.131454	13.267574	13.018520	13.134297	...	1.239131e-16	5.648860e-17	2.775161e-19	6.395538e-19	5.618435e-19	4.977891e-19	1.078377e-18	4.220456e-19	1.054331e-19	3.647871e-20

2 rows × 28 columns

Additional parameters

Additional arguments can be passed to the generator.

These are:

1. output_path
2. output_file
3. output_format
4. save_file

Three parameters: output_path, output_file, and output_format define the entire path of the resulting file.

The default output path is the current path. If the given output path does not exist, it will be created.

The default output file name is 'output_spectra'.

The default output format is the format of the input file (i.e. if the input file is a 'fits', then the output file will be a FITS file by default.), or CSV in any other case (DataFrame, ADQL query or list).

NOTE: If a file with the same path and name already exists, it will be AUTOMATICALLY OVERWRITTEN.

synthetic_photometry = generate(f, photometric_system=phot_system, output_path='/path/to', output_file='my_file', output_format='xml')

The additional parameter save_file is a boolean that tells the program whether to save the results or not. If 'output_file' is given but 'save_file' is set to False, a warning will be raised.

synthetic_photometry = generate(f, photometric_system=phot_system, output_file='my_file', save_file=False)

UserWarning: Argument output_file was given, but save_file is set to False. Set save_file to True to store the output of the function.