Skip to content

feat: Copy/archive input XML files into the output directory#4030

Open
kdrienCG wants to merge 39 commits into
developfrom
feature/kdrienCG/archiveInputDeck
Open

feat: Copy/archive input XML files into the output directory#4030
kdrienCG wants to merge 39 commits into
developfrom
feature/kdrienCG/archiveInputDeck

Conversation

@kdrienCG

@kdrienCG kdrienCG commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

(Previously #4003)

This PR proposes to archive the XML input deck to the output directory, so that every set of results is accompanied by the exact input that produced it.

When GEOS is run with the new archive command line option (-a LEVEL), all XML input files (and every files that are included via the <Included> XML tag) are flattened into a single file in a timestamped directory.

Following suggestions from @rrsettgast and @bd713, the XML input files are "flattened" into a single XML file.

The following command line argument is added to specifiy the level of archiving wanted:

-a, --archive,  Archiving strategy's level:
                  0 = no archiving,
                  1 = XML inputs only (default),
                  2 = XML inputs and the XSD schema

Given a run like:

geosx -i foo.xml -o OUTPUT/ -a 2

The following is created:

.
└─ OUTPUT/   
   └─ archive_inputFiles/
      └─ 20260320_103034/
         ├─ input.xml
         └─ schema.xsd

Previous proposition

This PR proposes to automatically archive the XML input deck to the output directory, so that every set of results is accompanied by the exact input that produced it.

When GEOS is run with an output directory specified (-o <dir>), all XML input files (and every files that are included via the <Included> XML tag) are copied into the output directory with a timestamp.


Given a run like:

geosx -i foo.xml -o OUTPUT/

The following is created:

.
└─ OUTPUT/   
   └─ archive_inputFiles/
      └─ 20260320_103034/
         ├─ foo.xml
         │
         ├─ include1.xml
         │
         └─ some_subdir/
            └─ include2.xml

When included files are "behind" the input tree of the main XML, the following structure is proposed:

.
└─ OUTPUT/   
   └─ archive_inputFiles/
      └─ 20260320_103034/
         ├─ foo.xml
         │
         ├─ __one_level_behind_include.xml
         │
         ├─ __one_level_behind_folder/           
         |  └─ baz.xml
         │
         └─ ____two_level_behind_folder/      
            └─ buzz.xml

Files outside the input tree (reached via ../) are prefixed with __ for every ../ from foo.xml.

kdrienCG and others added 20 commits March 20, 2026 09:23
Rename the archive output directory "archive_inputFiles" instead of "inputFiles".
This prevents the archived XML files to unintentionally overwrite the standard "inputFiles" in GEOS/ when running with `-o .` where '.' is GEOS/ location
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
ProblemManager::generateDocumentation() used to create the XSD schema has undesired side-effects on the Problem. To mitigate this, the archiving has been moved after the basicSetup() in the main.cpp, and creates an isolated ProblemManager (like the tests) that will not propagate the side-effects, and make us able to copy the XSD schema to the archive.
@kdrienCG kdrienCG self-assigned this Apr 16, 2026
@kdrienCG kdrienCG added type: feature New feature or request type: new A new issue has been created and requires attention labels Apr 16, 2026
@kdrienCG kdrienCG added the ci: run integrated tests Allows to run the integrated tests in GEOS CI label Apr 16, 2026
@kdrienCG kdrienCG marked this pull request as ready for review April 16, 2026 09:17
@kdrienCG kdrienCG removed the type: new A new issue has been created and requires attention label Apr 22, 2026
@kdrienCG kdrienCG marked this pull request as draft April 27, 2026 06:31
@herve-gross

Copy link
Copy Markdown
Contributor

@bd713 moving this to the review queue

Prefer local copy first
@kdrienCG kdrienCG marked this pull request as ready for review May 18, 2026 09:03

@bd713 bd713 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @kdrienCG.

Could you also update the GEOS documentation accordingly?
Typically QuickStart.rst for the new CLI flag, and maybe worth a brief description under Advanced XML Features.

Comment thread src/coreComponents/fileIO/Outputs/ArchiveInputDeck.cpp Outdated
Comment thread src/coreComponents/dataRepository/xmlWrapper.cpp Outdated
Comment thread src/coreComponents/common/initializeEnvironment.hpp Outdated
Comment thread src/coreComponents/mainInterface/initialization.cpp
Comment thread src/coreComponents/common/initializeEnvironment.hpp Outdated
@kdrienCG kdrienCG added the flag: no rebaseline Does not require rebaseline label May 22, 2026
@kdrienCG kdrienCG requested review from bd713 and jhuang2601 May 28, 2026 12:09

@MelReyCG MelReyCG left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there is no unitTest, how did you test this feature?

Comment thread src/main/main.cpp Outdated
{
GEOS_WARNING( GEOS_FMT( "Failed to copy XSD schema to archive '{}': {}",
destination.string(), ec.message() ) );
return;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you want to continue; here?

return timestampStream.str();
}

void stripMetadataAttributes( xmlWrapper::xmlNode node )

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure that you don't break something here? xmlWrapper::xmlNode is a wrapper around internal pugi data, I'm afraid you remove xmlWrapper::filePathString & xmlWrapper::charOffsetString definitively. Does it affect the line logging feature in GEOS_ERROR/GEOS_WARNING?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stripMetadataAttributes() runs on a copy/separate XML doc (flatDoc). It should not affect the XML document used for input parsing

string_array const & xmlTagOrder,
integer const level )
{
if( level == 0 || inputFileNames.empty() || outputDirectory.empty() )

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if( level == 0 || inputFileNames.empty() || outputDirectory.empty() )
if( level <= 0 || inputFileNames.empty() || outputDirectory.empty() )

?

{

/**
* @brief Archive the XML input deck (and optionally the XSD schema) into the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a note that it should not be called on multiple ranks

@kdrienCG kdrienCG requested a review from joshua-white as a code owner June 24, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: run code coverage enables running of the code coverage CI jobs ci: run CUDA builds Allows to triggers (costly) CUDA jobs ci: run integrated tests Allows to run the integrated tests in GEOS CI flag: no rebaseline Does not require rebaseline type: feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants