Skip to content

USER_MANUAL.md

Latest commit

 

History

History
684 lines (512 loc) · 23.5 KB

USER_MANUAL.md

File metadata and controls

684 lines (512 loc) · 23.5 KB
Raw

ExaGeoStatCPP User Manual

Content

  1. ExaGeoStatCPP v1.1.0
  2. Configurations of the software
  3. Building ExaGeoStatCPP
  4. Arguments
  5. List of Descriptors
  6. Supported operations
  7. Global Climate Emulator
  8. Manuals
  9. Contributing

ExaGeoStatCPP

Current Version of ExaGeoStatCPP: 2.0.0

Supported Operations:

  1. (Data Generation): Generating large geospatial synthetic datasets using dense, Diagonal Super-Tile (DST) and Tile Low-Rank (TLR) approximation techniques.
  2. (Data Modeling): Modeling large geospatial datasets on dense, DST and TLR approximation techniques through the Maximum likelihood Estimation (MLE) operation.
  3. (Data Prediction): Predicting missing measurements on given locations using dense, DST, and TLR approximation techniques.
  4. (MLOE/MMOM): Computing the Mean Loss of Efficiency (MLOE), Mean Misspecification of the Mean Square Error (MMOM), and Root mean square MOM (RMOM) to describe the prediction performance over the whole observation region.
  5. (Fisher Information Matrix (FIM)): Quantifying the information content that a variable x carries about a parameter $\theta$ within a Gaussian distribution.

Supported Covariance Functions:

  1. Univariate Matérn (Gaussian/Stationary)
  2. Univariate Matérn with Nugget (Gaussian/Stationary)
  3. Flexible Bivariate Matérn (Gaussian/Stationary)
  4. Parsimonious Bivariate Matérn (Gaussian/Stationary)
  5. Parsimonious trivariate Matérn (Gaussian/Stationary)
  6. Univariate Space/Time Matérn (Gaussian/Stationary)
  7. Bivariate Space/Time Matérn (Gaussian/Stationary)
  8. Tukey g-and-h Univariate Matérn (non-Gaussian/Stationary)
  9. Tukey g-and-h Univariate Power Exponential (non-Gaussian/Stationary)

To add your kernel, please refer to Contribution Guidelines

Programming models:

  1. MPI
  2. Task-based programming models

External libraries:

  1. NLOPT https://nlopt.readthedocs.io/en/latest/
  2. GSL https://www.gnu.org/software/gsl/
  3. HWLOC https://www.open-mpi.org/projects/hwloc/
  4. StarPU dynamic runtime system https://starpu.gitlabpages.inria.fr/
  5. HCORE https://github.com/ecrc/hcore
  6. HiCMA https://github.com/ecrc/hicma
  7. Stars-H https://github.com/ecrc/stars-h
  8. Chameleon https://gitlab.inria.fr/solverstack/chameleon

Project Hierarchy

  • cmake A directory contains essential CMake modules that facilitate the importation and location of required dependencies.
  • docs A directory contains all the necessary documents.
  • examples A directory contains a comprehensive collection of demo code that illustrates the framework's application and demonstrates its features and capabilities.
  • inst A directory contains all the system's header files, mirroring the structure of the src directory.
  • man A directory contains all the R functions documentation.
  • scripts A directory contains benchmarking scripts.
  • src A directory contains all the source files.
  • tests A directory contains all the test files and follows the same structure as the src folder.
  • clean_build.sh A script is designed to compile the software tests once all dependencies are installed, and it is set to build everything by default.
  • CMakeLists.txt The top-level CMake file to configure the build system.
  • configure A Script used to generate the building system inside a 'bin' directory.

Configurations

  • Run the help of configure to know the needed arguments for your specific options.

    ./configure -h

  • To enable R interface, add -r disabled by default.

  • To enable support of HiCMA, add -H disabled by default.

  • To enable examples, add -e enabled by default.

  • To enable tests, add -t disabled by default.

  • To enable heavy tests, add -T disabled by default.

  • To enable CUDA, add -c disabled by default.

  • To enable MPI, add -m disabled by default.

  • To enable verbose output, add -v disabled by default.

  • To change the installation path of the dependencies, use -i <installation/path> the default is project_path/installdir/_deps/ on Unix systems.

  • To enable packaging system for distribution, add -p disabled by default.

  • To enable showing code warnings, add -w disabled by default.

  • To manually set mkl as blas vendor, add --use-mkl. MKL is required as blas vendor and it's automatically detected but in some environments it need to be manually set.

  • To enable PaRSEC as a runtime system, add --use-parsec, StarPU is enabled by default.

  • To enable climate emulator examples (Mean Trend Removal + Climate Emulator), add --climate-emulator disabled by default. Note: Climate Emulator requires --use-parsec. Mean Trend Removal works with both StarPU and PaRSEC.

Building

  • Run the help of clean_build.sh to know additional argument options.

    ./clean_build.sh -h

  • Run clean_build.sh to build the project.

    ./clean_build.sh

  • To enable the installation of the project, add -i disabled by default.

  • To enable verbose printing, add -v disabled by default.

  • To enable building with a specific number of threads, add -j <thread_number> running with maximum number of threads by default.

Arguments

These are the arguments that you can specify when running any C++ example. Please note that the arguments are not case-sensitive, and you can use variations such as -, _, or capitalized forms for the same argument name. For example, MaxRank, max-rank, max_rank, Max_Rank ,Maxrank , MaxrAnk and Max-Rank are all considered equivalent.

  • {Mandatory} To set the problem size (N)

      --N=<problem_size>

  • {Mandatory} To set the kernel

      --kernel=<supported_kernel>

  • {Mandatory} To set the dense tile size in the case of Chameleon

      --dts=<value>

  • {Mandatory} To set the low tile size in case of HiCMA

      --lts=<value>

  • {Optional} To set the dimension, the default is 2D

      --dimension=<2D/3D/ST>

  • {Optional} To set the p grid, the default is 1

      --p=<value>

  • {Optional} To set the q grid, the default is 1

      --q=<value>

  • {Optional} To set the time slot, the default is 1

      --time_slot=<value>

  • {Optional} To set the computation, the default is dense

      --computation=<dense/tlr/dst>

  • {Optional} To set the precision, the default is double

      --precision=<single/double>

  • {Optional} To set the number of cores, the default is 1

      --cores=<value>

  • {Optional} To set the number of GPUs, the default is 0

      --gpus=<value>

  • {Optional} To set the number of unknown observations to be predicted, the default is 0

      --Zmiss=<value>

  • {Optional} To set the path of the observation file

      --observations_file=<path/to/file>

  • {Optional} To set the max rank, the default is 1

      --max_rank=<value>

  • {Optional} To set the lower bounds of optimization

      --olb=<value:value:....:value>

  • {Optional} To set the upper bounds of optimization

      --oub=<value:value:....:value>

  • {Optional} To set the initial theta

      --initial_theta=<value:value:....:value>

  • {Optional} To set the target theta

      --ttheta=<value:value:....:value>

  • {Optional} To set the estimated theta

      --estimated_theta=<value:value:....:value>

  • {Optional} To set the seed value, the default is 0

      --seed=<value>

  • {Optional} To set the verbose value, the default is standard

      --verbose=<quiet/standard/detailed>

  • {Optional} To set the path of log files to be written, the default is ./exageostat-cpp/synthetic_ds/

      --log_path=<path/to/file>

  • {Optional} To enable reading a CSV file containing real data, if not entered the default is the generation of synthetic data

      --data_path=<path/to/file>

  • {Optional} To enable out-of-core (OOC), the default is OFF

      --OOC

  • {Optional} To enable approximation mode, the default is ON

      --approximation_mode

  • {Optional} To enable writing log files, the default is OFF

      --log

Mean Trend Removal and Climate Processing Arguments

  • {Mandatory for Mean Trend Removal} To set the number of longitude points

      --lon=<value>

  • {Mandatory for Mean Trend Removal} To set the latitude band index

      --lat=<value>

  • {Mandatory for Mean Trend Removal} To set the starting year for data processing

      --startyear=<value>

  • {Mandatory for Mean Trend Removal} To set the ending year for data processing

      --endyear=<value>

  • {Mandatory for Mean Trend Removal} To set the path to forcing data file

      --forcing-data-path=<path/to/file>

  • {Optional} To set the results/output directory path

      --resultspath=<path/to/directory>

  • {Optional} To set the starting theta value for MLE, default is 0.9

      --starting-theta=<value>

  • {Optional} To set the lower bound for optimization, default is 0.001

      --lb=<value>

  • {Optional} To set the upper bound for optimization, default is 0.95

      --ub=<value>

  • {Optional} To set the optimization tolerance, default is 7

      --tolerance=<value>

  • {Optional} To set the maximum MLE iterations, default is 30

      --max-mle-iterations=<value>

  • {Mandatory for Mean Trend Removal} To enable Mean Trend Removal mode

      --mean-trend-removal

  • {Mandatory for Climate Emulator} To set the number of objects

      --ObjectsNumber=<value>

  • {Mandatory for Climate Emulator} To set the number of time points to process (z_*.csv files)

      --timeslot=<value>

  • {Optional} To set the diagonal addition value for regularization, default is 0

      --add-diagonal=<value>

  • {Optional} To set the accuracy threshold for low-rank approximation (0 = full rank), default is 0

      --Accuracy=<value>

  • {Optional} To set the band size for dense computations (double precision), default is 0

      --banddensedp=<value>

  • {Optional} To set the HNB parameter for matrix operations, default is 0

      --hnb=<value>

  • {Optional} To set the band size for dense operations (used in pipeline), default is 1000

      --band-dense=<value>

Full Pipeline Arguments

  • {Mandatory for Pipeline} To set the total number of latitude bands to process

      --lats=<value>

  • {Optional} To set the number of MPI processes per job, default is 2

      --mpi-processes=<value>

  • {Optional} To set the number of parallel jobs, default is 10

      --parallel-jobs=<value>

  • {Optional} To enable Climate Emulator after Mean Trend Removal

      --run-climate-emulator

List of Descriptors

Covariance Matrix Descriptors

  1. DESCRIPTOR_C: Covariance matrix C descriptor
  2. DESCRIPTOR_C11: Covariance Matrix C11 descriptor.
  3. DESCRIPTOR_C12: Covariance Matrix C21 descriptor.
  4. DESCRIPTOR_C21: Covariance Matrix C12 descriptor.
  5. DESCRIPTOR_C22: Covariance Matrix C22 descriptor.

Measurements Matrix Descriptors

  1. DESCRIPTOR_Z: Measurements Z descriptor.
  2. DESCRIPTOR_Z_COPY: A copy of Measurements Z descriptor.
  3. DESCRIPTOR_Z_OBSERVATIONS: Observed Measurements Z descriptor.
  4. DESCRIPTOR_Z_Actual: Actual Measurements Z descriptor.
  5. DESCRIPTOR_Z_MISS: Missing Measurements Z descriptor.

Measurements Sub-Matrix Descriptors

  1. DESCRIPTOR_Z_1: Measurements Z1 sub-matrix descriptor.
  2. DESCRIPTOR_Z_2: Measurements Z2 sub-matrix descriptor.
  3. DESCRIPTOR_Z_3: Measurements Z3 sub-matrix descriptor.

Dot Product Descriptors

  1. DESCRIPTOR_PRODUCT: Dot product descriptor.
  2. DESCRIPTOR_PRODUCT_1: Dot product descriptor.
  3. DESCRIPTOR_PRODUCT_2: Dot product descriptor.
  4. DESCRIPTOR_PRODUCT_3: Dot product descriptor.

Determinant Descriptors

  1. DESCRIPTOR_DETERMINANT: Determinant descriptor.

Mean Square Prediction Error Descriptors

  1. DESCRIPTOR_MSPE: Mean Square Error descriptor.

HiCMA Descriptors

  1. DESCRIPTOR_CRK: HiCMA descCrk descriptor.
  2. DESCRIPTOR_C12RK: HiCMA descCrk descriptor.
  3. DESCRIPTOR_C22RK: HiCMA descCrk descriptor.
  4. DESCRIPTOR_CD: HiCMA descCD descriptor.
  5. DESCRIPTOR_C12D: HiCMA descCD descriptor.
  6. DESCRIPTOR_C22D: HiCMA descCD descriptor.
  7. DESCRIPTOR_CUV : HiCMA descCUV descriptor
  8. DESCRIPTOR_C12UV : HiCMA descCUV descriptor
  9. DESCRIPTOR_C22UV : HiCMA descCUV descriptor

Supported Operations

Provide Arguments

To use any operations, you must initially supply the necessary arguments to the operation via the Configurations module. This program is configured by default through a json file of default values. You can either change the configuration arguments in the json file or provide them through command line. There are two methods available for setting your arguments:

  1. Provide your arguments with the command line, this overwrites the json configuration.
    // Create a new configuration object.
    Configurations configurations;
    // Initialize the arguments with the provided command line arguments
    configurations.InitializeArguments(argc, argv);
  1. Set your arguments manually in the code.
    Configurations synthetic_data_configurations;
    synthetic_data_configurations.SetProblemSize(10);
    synthetic_data_configurations.SetKernelName("BivariateSpacetimeMaternStationary");
    synthetic_data_configurations.SetPrecision(exageostat::common::double);

Initialize and Finalize the Hardware class

To use any operations, you must initialize the hardware by selecting the number of CPUs and/or GPUs.

// Initialize an instance of the hardware
auto hardware = ExaGeoStatHardware(computation, number of cores, number of gpus, p, q);

// Other code goes here

// Finalize the hardware instance.
hardware.FinalizeHardware()

The subsequent arguments are as follows:

  • computation: Specifies the computation mode for the solver.
  • number of cores: Indicates the number of CPU cores to be used for the solver.
  • number of gpus: Specifies the number of GPUs to be used for the solver.
ExaGeoStat R Interface
hardware <- new(Hardware, computation, number of cores, number of gpus, p-grid, q-grid);
hardware$finalize_hardware()

First arguement represents the name of the R class that wrapps its correponding C++ class, the rest of arguments are the same as the C++ version

Types of Data

ExaGeoStatCPP can be used with 2 types of data:

  • Synthetic data i.e. generated by the software according to the user arguments.
  • Real data e.g. data from satellite imagery or weather sensors. Real data can be used to train the software to predict the values of new data better.

Using Synthetic Data

Here we generate the data to be used by providing the needed arguments with the Configurations module, and then using the following code:

// Create a new ExaGeoStat data that holds the locations and descriptors data.
std::unique_ptr<ExaGeoStatData<double>> data;

// Generate data by passing your arguments through the configurations, hardware, 
//and container of the data, which will be filled with the newly generated data.
ExaGeoStat<double>::ExaGeoStatLoadData(configurations, data);

Using Real Data

Here we use existing data by providing the path to it:

  • The Data Path must be passed to Configuration 
    data_path <- <path/to/file>

And then using the following code:

exageostat_data <- simulate_data(kernel=kernel, initial_theta=initial_theta, problem_size=problem_size, dts=dts, dimension=dimension, data_path=data_path)

Location Getters

ExaGeoStat support 2D and 3D spatial locations, and therefore we have getters for X, Y and Z coordinates.

X-coordinate Getter

double *locations_x = exageostat_data->GetLocations()->GetLocationX();

Y-coordinate Getter

double *locations_y = exageostat_data->GetLocations()->GetLocationY();

Z-coordinate Getter

double *locations_z = exageostat_data->GetLocations()->GetLocationZ();
ExaGeoStat R Interface
locations <- get_locations(data=exageostat_data)

Returns all the locations values. The subsequent arguments are as follows:

  • exagostat_data: pointer to ExaGeoStatData object containing the spatial data.

Descriptive Z Values Getter

This function is used to retrieve descriptive Z values from ExaGeoStat data based on type of descriptor.

//in this example, we use a chameleon descriptor. Similar code can be used for hicma descriptor.
DescriptorType descriptor_type = CHAMELEON_DESCRIPTOR;
void *descriptor = exageostat_data->GetDescriptorData()->GetDescriptor(descriptor_type, DESCRIPTOR_Z).chameleon_desc;
double *desc_Z_values = exagostat_data->GetDescriptorData()->GetDescriptorMatrix(descriptor_type, descriptor);

The used variables are as follows:

  • descriptor_type: enum denoting the descriptor type,e.g. CHAMELEON_DESCRIPTOR,HICMA_DESCRIPTOR.
  • exagostat_data: pointer to ExaGeoStatData object containing the spatial data.
  • desc_Z_values: pointer to descriptor matrix.
ExaGeoStat R Interface
desc_Z_values <- get_Z_measurement_vector(data=exageostat_data, type="chameleon")

The subsequent arguments are as follows:

  • exagostat_data: pointer to ExaGeoStatData object containing the spatial data.
  • type: string specifying the type of descriptor value to retrieve.

Data Modeling

To use data modeling, you have to do this operation.

//You have to pass your arguments through the configurations, your hardware, and your data.
ExaGeoStat<double>::ExaGeoStatDataModeling(hardware, configurations, data, z_matrix);
ExaGeoStat R Interface
estimated_theta <- model_data(matrix=z_value, x=locations_x, y=locations_y, kernel=kernel, dts=dts, dimension=dimension,lb=lower_bound, ub=upper_bound, mle_itr=10, computation=computation, band=1)

Or

estimated_theta <- model_data(data=exageostat_data, kernel=kernel, dts=dts, dimension=dimension,lb=lower_bound, ub=upper_bound, mle_itr=10)

Data Prediction

//You have to pass your arguments through the configurations, your hardware, and your data.
ExaGeoStat<double>::ExaGeoStatPrediction(configurations, data, z_matrix);
ExaGeoStat R Interface
predict_data(train_data=list(locations_x, locations_y, locations_z, z_value), test_data=list(test_x, test_y, test_z), kernel=kernel, dts=dts, estimated_theta=estimated_theta)

Fisher Function

  1. Pass the fisher arguments to the Configurations.
--fisher
  1. Call the Data Prediction function.
// you have to pass your arguments through the configurations, your hardware and your data.
ExaGeoStat<double>::ExaGeoStatPrediction(configurations, data, z_matrix);
ExaGeoStat R Interface
fisher_matrix <- fisher(train_data=list(locations_x, locations_y, z_value), test_data=list(test_x, test_y), kernel=kernel, dts=dts, estimated_theta=estimated_theta)

MLOE-MMOM Function

  1. Pass the MLOE-MMOM arguments to the Configurations.
--mloe-mmom
  1. Call the Data Prediction function.
// you have to pass your arguments through the configurations, your hardware and your data.
ExaGeoStat<double>::ExaGeoStatPrediction(configurations, data, z_matrix);
ExaGeoStat R Interface
result_mloe_mmom = mloe_mmom(train_data=list(locations_x, locations_y, z_value), test_data=list(test_x, test_y), kernel=kernel, dts=dts, estimated_theta=estimated_theta, true_theta=true_theta)

IDW Function

  1. Pass the IDW arguments to the Configurations.
--idw
  1. Call the Data Prediction function.
// you have to pass your arguments through the configurations, your hardware and your data.
ExaGeoStat<double>::ExaGeoStatPrediction(configurations, data, z_matrix);
ExaGeoStat R Interface
idw_error = idw(train_data=list(locations_x, locations_y, z_value), test_data=list(test_x, test_y), kernel=kernel, dts=dts, estimated_theta=estimated_theta, test_measurements=test_measurements)

Global Climate Emulator

ExaGeoStatCPP provides specialized tools for climate data analysis through two main components:

Mean Trend Removal - Data Preprocessing

Purpose: Removes mean trends from climate time series data and produces normalized residuals for further analysis.

Runtime: Works with both StarPU and PaRSEC.

Configuration:

# With StarPU (default)
./configure -e --climate-emulator

# With PaRSEC
./configure -e --climate-emulator --use-parsec

Basic Usage:

mpirun -n 2 ./bin/examples/mean-trend-removal/Example_Mean_Trend_Removal \
  --kernel=trend_model \
  --lon=144 --lat=0 --dts=720 \
  --startyear=2020 --endyear=2020 \
  --data-path=/path/to/ERA_data/ \
  --forcing-data-path=/path/to/forcing_new.csv \
  --resultspath=/path/to/results/ \
  --starting-theta=0.9 \
  --lb=0.001 \
  --ub=0.95 \
  --tolerance=7 \
  --max-mle-iterations=30 \
  --mean-trend-removal \
  --cores=4 \
  --gpus=0 \
  --p=1 \
  --q=1

Outputs:

  • z_*.csv files: Normalized residuals (one file per time point)
  • params.csv: Optimized parameters for each location

Climate Emulator - Statistical Modeling

Purpose: Builds statistical models from Mean Trend Removal residuals to emulate climate behavior.

Runtime: Requires PaRSEC (StarPU not supported).

Configuration:

./configure -e --climate-emulator --use-parsec

Prerequisites:

  1. Mean Trend Removal output files (z_*.csv and params.csv)
  2. Auxiliary files matching your grid size: <dts>_Et1.csv, <dts>_Et2.csv, <dts>_Ep.csv

Basic Usage:

./bin/examples/climate-emulator/Example_Climate_Emulator \
  --N=5184 \
  --dts=72 \
  --timeslot=4000 \
  --ObjectsNumber=72 \
  --cores=1 \
  --add-diagonal=10 \
  --Accuracy=0 \
  --banddensedp=1000 \
  --hnb=300 \
  --verbose=detailed \
  --gpus=0 \
  --data_path=/path/to/results/

Key Parameters:

  • N: Spatial problem size (typically dts², e.g., N=5184 for dts=72)
  • timeslot: Number of time points to process (z_*.csv files)
  • data_path: Directory containing Mean Trend Removal outputs

Full Pipeline Script

Purpose: Automates processing of multiple latitude bands in parallel.

Script: FullPipeline.sh

Mean Trend Removal Processing:

./FullPipeline.sh \
  --lats=720 \
  --lon=1440 \
  --startyear=2000 \
  --endyear=2022 \
  --data-path=/path/to/ERA_data/ \
  --forcing-data-path=/path/to/forcing_new.csv \
  --resultspath=/path/to/results/ \
  --mpi-processes=5 \
  --parallel-jobs=40 \
  --cores=5

Full Pipeline (Mean Trend Removal + Climate Emulator):

./FullPipeline.sh \
  --lats=720 \
  --lon=1440 \
  --startyear=2020 \
  --endyear=2022 \
  --data-path=/path/to/ERA_data/ \
  --forcing-data-path=/path/to/forcing_new.csv \
  --resultspath=/path/to/results/ \
  --mpi-processes=5 \
  --parallel-jobs=40 \
  --cores=5 \
  --run-climate-emulator \
  --N=518400 \
  --dts=720 \
  --timeslot=4000 \
  --ObjectsNumber=720 \
  --add-diagonal=10 \
  --Accuracy=0 \
  --band-dense=1000 \
  --hnb=1000 \
  --verbose=detailed \
  --gpus=0

Important Notes:

  • Climate Emulator requires PaRSEC runtime
  • Mean Trend Removal works with both StarPU and PaRSEC
  • Ensure parallel-jobs × cores ≤ available CPU cores
  • For detailed parameter descriptions, see the Arguments section

Manuals

Contributing

Contribution Guidelines