From 05b387c473f1f2ccd8e8a1af094068bab97215b3 Mon Sep 17 00:00:00 2001 From: Lance-Drane Date: Wed, 18 Feb 2026 14:49:14 -0500 Subject: [PATCH] update IPS portal documentation Signed-off-by: Lance-Drane --- doc/user_guides/ensembles.rst | 2 ++ doc/user_guides/jupyter.rst | 11 ++++++ doc/user_guides/portal_guides.rst | 57 ++++++++++++++++++++++++++++--- 3 files changed, 66 insertions(+), 4 deletions(-) diff --git a/doc/user_guides/ensembles.rst b/doc/user_guides/ensembles.rst index 844272b6..c4bc9c5c 100644 --- a/doc/user_guides/ensembles.rst +++ b/doc/user_guides/ensembles.rst @@ -15,6 +15,8 @@ Ensemble simulations are useful when you need to: The ensemble functionality automatically generates configuration files for each simulation instance, distributes the workload across available compute nodes, and manages the execution of all ensemble members. +If interaction with the `IPS Portal `_ is enabled, the ensemble system will also handle interaction with the IPS Portal automatically. + Method Signature ---------------- diff --git a/doc/user_guides/jupyter.rst b/doc/user_guides/jupyter.rst index e7f0fc30..5a5849f4 100644 --- a/doc/user_guides/jupyter.rst +++ b/doc/user_guides/jupyter.rst @@ -120,6 +120,17 @@ The IPS Portal will generate a cell prior to your own notebook which initializes - ``ips_analysis_api.get_child_data_not_ensembles()`` - get the child runid mapping as described above, but only use child runids NOT associated with ensembles. - ``ips_analysis_api.get_child_data_by_ensemble_names()`` - gets the child runid mapping as described above, but will only retrieve child runids associated with ensembles. You can further filter this by ensemble name by providing an optional list of component names and an optional list of ensemble names; for example, ``ips_analysis_api.get_child_data_by_ensemble_names(ensemble_names=['ensemble_name_1', 'ensemble_name_2'])`` will ONLY fetch the child runids associated with 'ensemble_name_1' and 'ensemble_name_2', but will search all components for this. +An example of the "generic IPS mapping" generated by these functions can look like this: + +.. code-block:: python + + { + 0.0: ['/path/to/data/file1.json'], + 1.0: ['/path/to/data/file2.json'], + 2.0: ['/path/to/data/file3.json', '/path/to/data/file4.json'], # note that there can be multiple files per timestamp + # ... + } + **IPS Notebook Analysis API Reference** .. autoclass:: doc.reference.portal_jupyter_api.ips_analysis_api_v1.IPSAnalysisApi diff --git a/doc/user_guides/portal_guides.rst b/doc/user_guides/portal_guides.rst index 68145f37..2a73ba92 100644 --- a/doc/user_guides/portal_guides.rst +++ b/doc/user_guides/portal_guides.rst @@ -17,15 +17,22 @@ time, and a descriptive comment. From there you can click on a Run ID to see the details of that run, including calls on components, data movement events, task launches and finishes, and checkpoints. -To use the portal include +To use the portal on your local cluster, include the following variables in your configuration file or as environment variables: .. code-block:: text - USE_PORTAL = True + # stable version PORTAL_URL = http://lb.ipsportal.production.svc.spin.nersc.org + # or, for the latest version + # PORTAL_URL = http://lb.ipsportal.development.svc.spin.nersc.org + # The API key is required for certain interactions with the portal, and will eventually become mandatory. This key should generally be set as an environment variable, and not saved to version control. + PORTAL_API_KEY = "YOUR_PORTAL_API_KEY" # change this + # To disable the portal even if PORTAL_URL is set, uncomment the next line. + # USE_PORTAL = false +NOTE: On shared clusters, i.e. Perlmutter, there will generally be specific files that you can source in Slurm scripts which will automatically configure the Portal credentials for you, so you can skip settings these variables yourself. Please see the appropriate project documentation for information on how to configure this. -The source code for the portal can be found one `GitHub +The source code for the portal can be found on `GitHub `_ and issues can be reported using `GitHub issues `_. @@ -91,7 +98,7 @@ like: child_conf['PARENT_PORTAL_RUNID'] = self.services.get_config_param("PORTAL_RUNID") This is automatically configured when running -``ips_dakota_dynamic.py``. +``ips_dakota_dynamic.py`` or when using the ``run_ensemble`` API. The child runs will not appear on the main runs list but will appear on a tab next to the events. @@ -102,3 +109,45 @@ The trace of the primary simulation will contain the traces from all the simulations: .. image:: child_runs_trace.png + +IPS-Framework APIs +------------------ + +Provided that the Portal has been enabled, the following APIs allow for interaction with the web portal: + +Events API +========== + +.. automethod:: ipsframework.services.ServicesProxy.send_portal_event + :noindex: + +This function can be used to send custom events to the Portal if the Portal is enabled; if the Portal is not enabled, the event will still be logged locally. + +The events API does not currently require an API key to utilize, but this is expected to change in the future. + +Jupyter API +=========== + +.. automethod:: ipsframework.services.ServicesProxy.initialize_jupyter_notebook + :noindex: + +.. automethod:: ipsframework.services.ServicesProxy.add_analysis_data_files + :noindex: + +All Jupyter APIs require that the Portal API key is set in order to utilize them. + +If calling either of these APIs when the Portal is disabled, a warning will be logged and the call will be skipped. + +Please see the `Jupyter `_ page for specifics on using the Jupyter APIs. + +Ensembles API +============= + +.. automethod:: ipsframework.services.ServicesProxy.run_ensemble + :noindex: + +If the Portal is enabled and the API key has been set, ``run_ensemble`` will automatically interact with the Portal to send appropriate files. + +If the Portal is disabled, ``run_ensemble`` will skip the Portal interaction, but will otherwise behave normally. + +Please see the `Ensembles `_ page for specifics on using the Ensembles API.