Visualisation in-situ with Paraview Catalyst

Visualisation in-situ is quite fashinable nowadays. Indeed, for exascale simulations, the expension of data generated by a scientific applicaitons will become too big to continue doing as we did : write raw data on the file system and do some post-processing afterwards. This is were in-situ visualisation come in place, visualise data as they are created and/or export pictures, videos from the simulation. Catalyst is a module of Paraview that allows us to do this.

Paraview Catalyst

Paraview is an open-source visualisation application developped by kitware.	Catalyst is Paraview component to do in-situ data analysis and visualisation

VTK

For the visualisation, Paraview relies on the Visualization Tookit (alias VTK) . As stated on their website,

VTK is an open-source, freely available software system for 3D computer graphics, image processing, and visualization. It supports a wide variety of visualization algorithms including scalar, vector, tensor, texture, and volumetric methods, as well as advanced modeling techniques such as implicit modeling, polygon reduction, mesh smoothing, cutting, contouring, and Delaunay triangulation.

In order to use Paraview Catalyst, your first step is to make a VTK Grid depending of your mesh type. I use VTKUnstructuredGrid for 3D meshes and VTKPolyData for 2D Meshes.

Software modification

Once you have a VTK mesh, you need to write 3 functions specific to your implementation : init(QString script), finalize(), and coProcess(double time, int time_step) and add them to your workflow as with this example.

init and finalize functions are self explaining. coProcess has be called each time your data changes and you want to update the visualisation. The type of visualisation work to do is based on a script given at runtime and passed to init. You can choose to export pictures of all or some parts of the simulation from a camera position and angle or to do in-situ visualisation. The key class allowing all of this to happen being VTKCPProcessor.

Below is my implementation of these 3 functions as an example.

void dtkMeshViewVTKCatalystPrivate::initCatalyst(QString script)
{
   if(!vtk_processor)
    {
        vtk_processor = vtkCPProcessor::New();
        vtk_processor->Initialize();
    }
    else  {
        vtk_processor->RemoveAllPipelines();
    }
    
    vtkNew<vtkCPPythonScriptPipeline> pipeline;
    pipeline->Initialize(script.toStdString().c_str());
    vtk_processor->AddPipeline(pipeline.GetPointer());
}

void dtkMeshViewVTKCatalystPrivate::finalizeCatalyst(void)
{
  if(vtk_processor)
    {
        vtk_processor->Delete();
        vtk_processor = nullptr;
    }
}

void dtkMeshViewVTKCatalystPrivate::coProcessCatalyst(double time, unsigned int time_step, bool last_time_step)
{
    vtkNew<vtkCPDataDescription> dataDescription;
    dataDescription->AddInput("input");
    dataDescription->SetTimeData(time, time_step);

    if(last_time_step == true)
    {
        dataDescription->ForceOutputOn();
    }

    if(vtk_processor) {
        if(vtk_processor->RequestDataDescription(dataDescription.GetPointer()) != 0)
        {
            updateAttributes();
            dataDescription->GetInputDescriptionByName("input")->SetGrid(mesh_actor->grid());
            vtk_processor->CoProcess(dataDescription.GetPointer());
        }
    }
}

You can see that if the visualisation work is happening, I update my VTKGrid calling the function updateAttributes().

Catalyst script

The script used to define visualisation parameters such as camera position, when to export pictures, do or not do in-situ is a python script. You can write it or use Paraview to generate it for you. If you want to do the latest, here is what you have to do:

1. load paraview catalyst plugin in Parview, go to Tools/Manage_plugins and load CatalystScriptGeneratorPlugin.
2. load representative dataset is has to contains all the fields/attributes you will want to visualize (fill them with 0)
3. set your layout(s) position your camera, choose the Color Map, define slices, ..
4. write the script Wo in CoProcessing/Export State, follow instructions

Note that if you are doing in-situ visualization, your color-map and co won’t be taken into consideration!

We are now ready to launch the simulation!  

Simulation and Visualisation on same machine

Here are the steps to follow for in-situ visualisation and simulation on the same machine:

1. in Paraview do Catalyst/Connect, use default port 22222
2. pause simulation before launching it Catalyst/Pause_simulation otherwise the simulation might end before the creation of the pipeline and we won’t see anything.
3. launch your simulation simu It will automatically connect to paraview
4. in Paraview, you should see a small round icon appear in the pipeline browser. Click on it to extract the raw data sent from the simulation.
5. do Catalyst/Continue to launch the simulation. Note that you can also set breakpoints at certain timesteps or certain time.  

Simulation on cluster + rendering on GPU cluster node + visualisation on laptop

In order to do simulation, rendering, and visualisation on three different machines, here is what you have to do:

1. On GPU-Rendering node launch a pvserver : ` mpirun -np 2 pvserver -display :0.0 : -np 2 pvserver -display :0.1 `
2. On visualisation laptop start Paraview and do
   1. File/connect nefgpuXX
   2. Catalyst/Connect
   3. Catalyst/pause simulation
3. on the cluster
   1. edit the Catalyst script to set the name of the host with the pvserver: it’s the last line of the script coprocessor.DoLiveVisualization(datadescription, "nefgpuXX", 22222)
   2. launch your application : mpirun -np XX ./my_app
4. On visualisation laptop do the visualisation

Here is a video showing these steps

If yout want more informations on Paraview Catalyst, here is their wiki and the official user manual