ONYX - 9.0 - Exploitation - Guide d'exploitation ONYX Server sur Linux/en

Différence entre versions

De MappingDoc
(Page créée avec « =====<u> a "scanfolder" robot crashed </u>===== When starting a robot, a file with the same name as the robot and a ".ID" extension is created, as well as a file with the... »)
(Page créée avec « <u>Spooler start-up check list</u> *'''map_daemon''' process must be stopped *The '''map_daemon.ID''' file must be deleted *The '''map_daemon port''' (by default 2000) mus... »)
 
(33 révisions intermédiaires par 4 utilisateurs non affichées)
Ligne 6 : Ligne 6 :
 
This only regards ONYX Server. The administration of Apache web server (stop / restart) is not described in this article.
 
This only regards ONYX Server. The administration of Apache web server (stop / restart) is not described in this article.
  
=== General operations ===
+
=== General Operations ===
 
==== Selecting the Mapping instance ====
 
==== Selecting the Mapping instance ====
 
Several different ONYX Server instances can be installed on the same Unix/Linux server. This way, specifying the Mapping environment is required before entering any command. This is done by changing the MAPPING_PATH environment variable.
 
Several different ONYX Server instances can be installed on the same Unix/Linux server. This way, specifying the Mapping environment is required before entering any command. This is done by changing the MAPPING_PATH environment variable.
Ligne 25 : Ligne 25 :
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf
  
=== Log files ===
+
=== Log Files ===
 
==== General information on log files ====
 
==== General information on log files ====
 
Most Mapping log files are not in text format, they thus cannot be edited as is.
 
Most Mapping log files are not in text format, they thus cannot be edited as is.
Ligne 64 : Ligne 64 :
  
 
==== Log files of the jobs in the spooler ====
 
==== Log files of the jobs in the spooler ====
Each job as its own log file. These files are located in the /apps/mapping/spool/global file. Their name starts with the letter L followed by the number of the job (MAP_JOBNUM).
+
Each job has its own log file. These files are located in the /apps/mapping/spool/global file. Their name starts with the letter L followed by the number of the job (MAP_JOBNUM).
 
 
 
=== Processes ===
 
=== Processes ===
Ligne 75 : Ligne 75 :
 
| map_daemon || Processes of the spooler which allow jobs to be managed in the queues.
 
| map_daemon || Processes of the spooler which allow jobs to be managed in the queues.
 
|-
 
|-
| map_lpd || Recovery process of the jobs sent via LPR. This is a derivative of the map_daemon process.
+
| map_lpd || Recovery process of the jobs sent via LPR. This is a child of the map_daemon process.
 
|-
 
|-
| map_splf || Process which notably allows you to add a job in a queue. This is a derivative of the map_lpd process, it communicates with the map_daemon process via the port defined by the PORT_SOCKET_DAEMON parameter in the mapping.conf file. It can also be directly triggered in command line to manage queues or jobs.
+
| map_splf || Process which notably allows you to add a job in a queue. This is a child of the map_lpd process, it communicates with the map_daemon process via the port defined by the PORT_SOCKET_DAEMON parameter in the mapping.conf file. It can also be directly triggered in command line to manage queues or jobs.
 
|-
 
|-
| map_exec || Process which handles jobs in queues to apply processes to them (workflow, script execution, delivery to a printer…). This is a derivative of the  map_daemon process. There is one map_exec per queue being processed.
+
| map_exec || Process which handles jobs in queues to apply processes to them (workflow, script execution, delivery to a printer…). This is a child of the  map_daemon process. There is one map_exec per queue being processed.
 
|-
 
|-
| map_lpr || Process which sends a printing stream to a printer using the LPR protocol. This is a derivative of the map_exec process. It can also be directly triggered in command line.
+
| map_lpr || Process which sends a printing stream to a printer using the LPR protocol. This is a child of the map_exec process. It can also be directly triggered in command line.
 
|-
 
|-
| map_809 || Execution Process of a Workflow. This is a derivative of the  map_exec process.
+
| map_809 || Execution Process of a Workflow. This is a child of the  map_exec process.
 
|-
 
|-
 
| mapcpysplf || Process which calls for the mapping of a document. It can be triggered in command line, in a script, in the workflow (map_809)…
 
| mapcpysplf || Process which calls for the mapping of a document. It can be triggered in command line, in a script, in the workflow (map_809)…
Ligne 100 : Ligne 100 :
 
=====<u> Starting the spooler</u> =====
 
=====<u> Starting the spooler</u> =====
  
The following command launches the map_daemon process as well as its derivative map_lpd
+
The following command launches the map_daemon process as well as its child map_lpd
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_daemon start
 
  /apps/mapping/bin/map_daemon start
  
<div class="mw-translate-fuzzy">
 
 
When starting the map_daemon process, a flag file which contains the process identifier of map_daemon process is created in the spooled file folder. The process cannot start because of it.
 
When starting the map_daemon process, a flag file which contains the process identifier of map_daemon process is created in the spooled file folder. The process cannot start because of it.
 
  /apps/mapping/spool/map_daemon.ID
 
  /apps/mapping/spool/map_daemon.ID
</div>
+
 
 +
<u>Spooler start-up check list</u>
 +
*'''map_daemon''' process must be stopped
 +
*The '''map_daemon.ID''' file must be deleted
 +
*The '''map_daemon port''' (by default 2000) must not be used
 +
*The '''map_lpd port''' (by default 515) must not be used
 +
 
  
 
=====<u> Stopping the spooler</u> =====
 
=====<u> Stopping the spooler</u> =====
  
The following command stops the map_daemon process as well as its derivative map_lpd
+
The following command stops the map_daemon process as well as its child map_lpd
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf
 
  /apps/mapping/bin/map_daemon stop
 
  /apps/mapping/bin/map_daemon stop
Ligne 131 : Ligne 136 :
 
  done
 
  done
  
When starting the map_scanfolder process, a flag file contenant le process identifier of the map_scanfolder process is created in temp folder. The process cannot start because of it. It needs to be deleted.
+
When starting the map_scanfolder process, a flag file which contains the process identifier of the map_scanfolder process is created in the temp folder. The process cannot start because of it. It needs to be deleted.
 
  /apps/mapping/temp/_apps_mapping_temp_nom_du_robot_map_scanfolder.ID
 
  /apps/mapping/temp/_apps_mapping_temp_nom_du_robot_map_scanfolder.ID
  
Ligne 187 : Ligne 192 :
 
* map_809
 
* map_809
  
It is worth noting that processes can become derivatives of the system process (PID=1) in case the process they originate from crashes, common characteristic: they are owned by "mapadmin".
+
It is worth noting that processes can become children of the system process (PID=1) in case the process they originate from crashes, common characteristic: they are owned by "mapadmin".
  
 
Example (adapt this depending on your system and its configuration):
 
Example (adapt this depending on your system and its configuration):
Ligne 212 : Ligne 217 :
 
  4656.pid
 
  4656.pid
  
En cas de crash du robot, ces fichiers ne sont pas supprimés et empêche le redémarrage du robot.
+
Were the robot to crash, these files would not be deleted and would prevent the robot from being restarted.
Par conséquent, pour permettre le redémarrage du robot, le fichier « .ID » correspondant au robot arrêté doit être supprimé du répertoire « /apps/mapping/temp »
+
To restart the robot, the ".ID" file which corresponds to the crashed robot must then be deleted from the " /apps/mapping/temp " folder
  
=== Opérations de maintenance ===
+
=== Maintenance ===
==== Nettoyage courant ====
+
==== Regular cleaning ====
Les commandes suivantes peuvent être planifiées très régulièrement. Dans l’idéal, au moins une fois par jour.
+
The following commands can be set to be executed regularly. Ideally, at least once a day.  
=====<u> Suppression des travaux terminés et arrivé à échéance</u> =====
+
=====<u> Deleting finished and expired jobs</u> =====
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_cron -date
 
  /apps/mapping/bin/map_cron -date
  
=====<u> Suppression des travaux non effectués (quel que soit leur statut), après 30 jours</u> =====
+
=====<u> Deleting non finished jobs (no matter their status), after 30 days</u> =====
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_cron -cleanspool -relative_date:30/0/0.  # jour/mois/années
 
  /apps/mapping/bin/map_cron -cleanspool -relative_date:30/0/0.  # jour/mois/années
  
=====<u> Suppression des fichiers de session web obsolètes</u> =====
+
=====<u> Deleting outdated web session files </u> =====
Fichiers context_menu.*, historique.*, et *.id du dossier /apps/mapping/temp
+
context_menu.*, history.*, and *.id files in the /apps/mapping/temp folder
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_cron -cleanid
 
  /apps/mapping/bin/map_cron -cleanid
  
=====<u> Suppression des fichiers temporaires de conversion XPS et de Workflow</u> =====
+
<div class="mw-translate-fuzzy">
Ces fichiers se trouvent dans le dossier temporaire /apps/mapping/temp.
+
=====<u> Deleting XPS conversion and Workflow temporary files </u> =====
Normalement, ils sont supprimés automatiquement. Mais en cas de problème (crash, interruption forcée…), ils peuvent subsister dans le dossier temp.
+
These files are located in the /apps/mapping/temp temporary folder.
Ce n’est donc pas une situation normale.
+
They are normally deleted automatically. In case of problem (crash, forced interruption…), they may remain in the temporary file.
 +
It is then not a normal situation.
 
  find /apps/mapping/temp -name "*cri.tmp"    -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*cri.tmp"    -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*ttf.tmp"    -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*ttf.tmp"    -mtime +2 -exec rm -f {} \;
Ligne 240 : Ligne 246 :
 
  find /apps/mapping/temp -name "*.[0-9]"      -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*.[0-9]"      -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*.[0-9][0-9]" -mtime +2 -exec rm -f {} \;
 
  find /apps/mapping/temp -name "*.[0-9][0-9]" -mtime +2 -exec rm -f {} \;
 +
</div>
  
==== Opérations de nettoyage et de rotations des logs ====
+
==== Log cleaning and rotation ====
Pour des raisons pratiques, les commandes suivantes ne doivent pas être planifiées plus d’une fois par jour. Cela afin de ne pas multiplier les archives créées, et éviter de perdre des derniers jours de logs. La fréquence idéale est de 2 fois par semaine, par exemple le lundi soir et le jeudi soir, avant le déclenchement des batchs de nuit.
+
For practical reasons, the following commands must not scheduled for more than once a day. The aim is to prevent increasing the number of archives created, thus preventing the loss of the logs of the last days. The ideal frequency is 2 times a week, for instance monday night and thursday night, before the night batches are trigerred.
=====<u> Archivage des logs</u> =====
+
=====<u> Archiving logs</u> =====
Cette commande permet de faire une conversion texte de tous les fichiers de logs, puis de les archiver dans un fichier zip horodaté.
+
This command allows you to convert any log file to text format and afterwards, to archive it in a zipped file which shows the date and time.
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT
 
  /apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT
  
Résultat :
+
Result:
 
  bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
 
  bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
 
  -rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
 
  -rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
 
  -rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
 
  -rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
  
=====<u> Rotation des logs des sorties standard et erreur</u> =====
+
=====<u> Log rotation of the standard outputs and errors </u> =====
 
  mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak  
 
  mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak  
 
  mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
 
  mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
  
=====<u> Suppression des archives de logs de plus de 30 jours</u> =====
+
=====<u> Deleting log archives that are older than 30 jours</u> =====
 
  find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;
 
  find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;
  
=====<u>Archivage des logs</u>=====
+
=====<u>Archiving logs</u>=====
Cette commande permet de faire une conversion texte de tous les fichiers de logs, puis de les archiver dans un fichier zip horodaté.
+
This command allows you to convert any log file to text format and afterwards, to archive it in a zipped file which shows the date and time.
 
export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
export MAPPING_PATH=/apps/mapping/conf/mapping.conf  
 
  /apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT
 
  /apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT
  
Résultat :
+
Result:
 
  bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
 
  bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
 
  -rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
 
  -rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
 
  -rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
 
  -rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
  
=====<u> Rotation des logs des sorties standard et erreur</u> =====
+
=====<u> Log rotation of the standard outputs and errors</u> =====
 
  mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak  
 
  mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak  
 
  mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
 
  mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
  
=====<u> Suppression des archives de logs de plus de 30 jours</u>=====
+
=====<u> Deleting log archives that are older than 30 days</u>=====
 
  find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;
 
  find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;
 +
 +
<div class="mw-translate-fuzzy">
 +
====Sample purge script====
 +
#Purge script example
 +
  #!/bin/bash
 +
    echo "------------- Initialization of the parameters necessary for the execution of the script ---------------------"
 +
      echo "Set environment"
 +
          export MAPPING_PATH=/apps/mapping/conf/mapping.conf
 +
      echo "Variable creation containing the path of the Mapping binaries"
 +
          PATH_BIN="/apps/mapping/bin"
 +
      echo "Creation of the other variables used in the script"
 +
          TIMEDATE=`date "+%Y_%m_%d_%H_%M_%S"`
 +
      echo "My variable PATH_BIN :  $PATH_BIN"
 +
      echo "MAPPING_PATH : $MAPPING_PATH"
 +
      echo "Variable creation and population set with the content of the mapping.conf file used by the script"
 +
          PATH_LOG_BACKUP=`$PATH_BIN/map_004 PATH_LOG_BACKUP`
 +
          PATH_BASE_MAPPING=`$PATH_BIN/map_004 PATH_BASE_MAPPING`
 +
      echo "My variable PATH_LOG_BACKUP : $PATH_LOG_BACKUP"
 +
      echo "My variable PATH_BASE_MAPPING : $PATH_BASE_MAPPING"
 +
      echo "My timedate : $TIMEDATE"
 +
  echo "-------------------------- Compression and archiving of the log files/folders ---------------------------------------------"
 +
      echo "Archiivng of the log folder /apps/mapping/spool/logs in to the log backup folder"
 +
          $PATH_BIN/map_cron -cleanlog -folder:$PATH_BASE_MAPPING/mapping_temp/log_temp
 +
      echo "Packaging of the content of the mapping log folder and mapping Log Custom folder"
 +
          tar -cvf $PATH_BASE_MAPPING/mapping_temp/log_temp/Archive_M-Connect_$TIMEDATE.tar $PATH_BASE_MAPPING/spool/mapping_log_custom
 +
          tar -cvf $PATH_BASE_MAPPING/log_archive/log_Archive_$TIMEDATE.tar $PATH_BASE_MAPPING/mapping_temp/log_temp
 +
      echo "Compression of the archive log folder"
 +
          gzip $PATH_BASE_MAPPING/log_archive/log_Archive_$TIMEDATE.tar
 +
  echo "--------------------- Clean of the log folder and mapping log custom folder --------------------------"
 +
      echo "Suppression du contenu du répertoire de log de M-Connect"
 +
          rm -f $PATH_BASE_MAPPING/spool/mapping_log_custom/*
 +
      echo "Suppression du contenu du répertoire de log temporaire"
 +
          rm -f $PATH_BASE_MAPPING/mapping_temp/log_temp/*
 +
      echo "clean of the saved spool files"
 +
          $PATH_BIN/map_cron -cleanspool -state:saved
 +
      echo "Clean of .id , .pid , historique , context_menu files of the mapping /apps/mapping/temp folder"
 +
          $PATH_BIN/map_cron -cleanid
 +
      echo "Clean of the temporally files older than 2 days of the /apps/mapping/temp folder"
 +
          find $PATH_BASE_MAPPING/temp -name "*cri.tmp" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "*ttf.tmp" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "compress_*" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "*.[0-9]" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "*.[0-9][0-9]" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "xps_*" -mtime +2 -exec rm -f {} \;
 +
          find $PATH_BASE_MAPPING/temp -name "tmp_report*" -mtime +2 -exec rm -f {} \;
 +
      echo "Suppression du contenu de plus de 30 jours du répertoire d'archivage /apps/mapping/log_archive"
 +
          find $PATH_BASE_MAPPING/log_archive -name "*" -mtime +30 -exec rm -f {} \;
 +
</div>

Version actuelle datée du 26 mai 2020 à 07:38

Autres langues :
English • ‎français

Introduction

This article provides you with examples of operations on ONYX Server under Linux. This needs to be adapted according to your internal processes.

This only regards ONYX Server. The administration of Apache web server (stop / restart) is not described in this article.

General Operations

Selecting the Mapping instance

Several different ONYX Server instances can be installed on the same Unix/Linux server. This way, specifying the Mapping environment is required before entering any command. This is done by changing the MAPPING_PATH environment variable.

In interactive mode: with the "mappingenv" command

-bash-4.2$ mappingenv
Which environment must be changed:
1 : Mapping_PROD   /apps/mapping/conf/mapping.conf 8002
Enter the name or number of the environment:1
bash-4.2$

The list of environments is is based on the /etc/mappingtab file

The mappingenv command is located in " /bin " by default

In a script: The MAPPING_PATH environment variable must be exported with the complete path of the mapping.conf file which corresponds to the desired Mapping environment.

export MAPPING_PATH=/apps/mapping/conf/mapping.conf

Log Files

General information on log files

Most Mapping log files are not in text format, they thus cannot be edited as is. There are two methods used to see them:

  • Via the Mapping Web interface
  • Using the /apps/mapping/bin/map_log_txt command


For more information on this command:

/apps/mapping/bin/map_log_txt --help

Display example of the content of the log file of the map_lpd:

/apps/mapping/bin/map_log_txt -log_file:/apps/mapping/spool/logs/map_lpd.log

Common log files

Most log files are located in the /apps/mapping/spool/logs folder.

Log file examples:

<name of the queue>.log Logs linked to queues (printer or entry points)
map_daemon.log General log of the spooler
map_lpd.log Log of the LPD server
map_lpr.log Log of the LPR Mapping client if called in a command line

To access it via the interface: "Main menu" / "View log"

Log files linked to standard outputs and errors

/apps/mapping/temp/stdout.txt
/apps/mapping/temp/stderr.txt

These are text files which can be edited as is without needing to be converted.

Log files of the jobs in the spooler

Each job has its own log file. These files are located in the /apps/mapping/spool/global file. Their name starts with the letter L followed by the number of the job (MAP_JOBNUM).  

Processes

Description of the main Mapping processes

The following processes are frequently seen when displaying the list of current processes. This is often done using the following command:

ps -eaf | grep map
map_daemon Processes of the spooler which allow jobs to be managed in the queues.
map_lpd Recovery process of the jobs sent via LPR. This is a child of the map_daemon process.
map_splf Process which notably allows you to add a job in a queue. This is a child of the map_lpd process, it communicates with the map_daemon process via the port defined by the PORT_SOCKET_DAEMON parameter in the mapping.conf file. It can also be directly triggered in command line to manage queues or jobs.
map_exec Process which handles jobs in queues to apply processes to them (workflow, script execution, delivery to a printer…). This is a child of the map_daemon process. There is one map_exec per queue being processed.
map_lpr Process which sends a printing stream to a printer using the LPR protocol. This is a child of the map_exec process. It can also be directly triggered in command line.
map_809 Execution Process of a Workflow. This is a child of the map_exec process.
mapcpysplf Process which calls for the mapping of a document. It can be triggered in command line, in a script, in the workflow (map_809)…
map_815UCS Mapping processus of a document (called by the mapcpysplf command).
map_mail Process which sends an e-mail with an attachment.
map_scanfolder Process linked to a robot which allows you to monitor files in a folder so as to launch a workflow processing on each of them.

Other processes with the prefix " map_ " can also be seen, notably if they are called via scripts or in the workflow.

Starting and stopping Mapping processes

Starting the spooler

The following command launches the map_daemon process as well as its child map_lpd

export MAPPING_PATH=/apps/mapping/conf/mapping.conf 
/apps/mapping/bin/map_daemon start

When starting the map_daemon process, a flag file which contains the process identifier of map_daemon process is created in the spooled file folder. The process cannot start because of it.

/apps/mapping/spool/map_daemon.ID

Spooler start-up check list

  • map_daemon process must be stopped
  • The map_daemon.ID file must be deleted
  • The map_daemon port (by default 2000) must not be used
  • The map_lpd port (by default 515) must not be used


Stopping the spooler

The following command stops the map_daemon process as well as its child map_lpd

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
/apps/mapping/bin/map_daemon stop

Caution: if "scanfolder" robots processing consists in sending a file to a queue of the spooler, stopping the map_daemon process will result in errors each time a file is processed. It is therefore essential to stop the "scanfolder" robots before stopping the map_daemon process.

Starting the "scanfolder" robots

The following command starts a "scanfolder" robot

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
/apps/mapping/bin/map_scanfolder -name:nom_du_robot

Example of a script to start all the robots:

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
LISTROBOT=`/apps/mapping/bin/map_scanfolder -listRobot | awk ' { print $2 } '`
for ROBOT in $LISTROBOT
do
/apps/mapping/bin/map_scanfolder -name:${ROBOT}
done

When starting the map_scanfolder process, a flag file which contains the process identifier of the map_scanfolder process is created in the temp folder. The process cannot start because of it. It needs to be deleted.

/apps/mapping/temp/_apps_mapping_temp_nom_du_robot_map_scanfolder.ID

Caution: deleting this .ID file should only be done if the robot is not being run. When deleting this file, launching another instance of the same robot is possible which causes concurrency conflicts.

Stopping the "scanfolder" robots

The following command stops a "scanfolder" robot

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
/apps/mapping/bin/map_scanfolder -stop -name:nom_du_robot

Example of a script to stop all the robots:

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
LISTROBOT=`/apps/mapping/bin/map_scanfolder -listRobot | awk ' { print $2 } '`
for ROBOT in $LISTROBOT
do
/apps/mapping/bin/map_scanfolder -stop -name:${ROBOT}
done

Processes to monitor

map_daemon and map_lpd

These 2 processes are services and need to be run incessantly. Because they are often forked, there may be several of these processes at some point.

-bash-4.1$ ps -eaf | grep map_
mapadmin 29551     1  1 Oct25 ?   04:00:29 /apps/mapping/bin/map_daemon start
mapadmin 27424 29551  0 Oct30 ?   00:00:00 /apps/mapping/bin/map_lpd -port:515 -nbfork:100
map_scanfolder

There are as many map_scanfolder processes as there are robots being executed. They can be identified with the "-name" option. These processes do not depend on one another, neither do they depend on the map_daemon process.

mapadmin   8271  1  0 17:02 ? 00:00:00 /apps/mapping/bin/map_scanfolder -name:test
httpd

To check that Apache runs smoothly, use the httpd process. Because this process is often forked, several lines can be seen in the list of processes.

-bash-4.2$ ps -eaf | grep httpd
apache    4590 11849  0 10:56 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
apache    4874 11849  0 08:23 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
apache    4876 11849  0 08:23 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
apache    4890 11849  0 08:23 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
root     11849     1  0 oct.15 ?       00:01:42 /usr/sbin/httpd -DFOREGROUND
apache   15641 11849  0 08:54 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND
apache   16301 11849  0 08:56 ?        00:00:00 /usr/sbin/httpd -DFOREGROUND

In case a Mapping process crashes

map_daemon crashed

1- Try stopping the Mapping spooler properly:

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
/apps/mapping/bin/map_daemon stop

2- Stop the active scanfolder processes properly (see starting and stopping Mapping processes)

3- Kill (kill -9) the other mapping processes which are still active:

  • map_daemon
  • map_lpd
  • map_exec
  • map_lpr
  • map_809

It is worth noting that processes can become children of the system process (PID=1) in case the process they originate from crashes, common characteristic: they are owned by "mapadmin".

Example (adapt this depending on your system and its configuration):

ps -f -U mapadmin | grep -v bash | grep -v "ps \-f" | grep -v "UID" | while read LINE
do
 PID=`echo $LINE | awk ' { print $2 } '`
 kill -9 $PID
done

4- Delete the flag file /apps/mapping/spool/map_daemon.ID

5- Start the Mapping spooler

/apps/mapping/bin/map_daemon start
map_lpd crashed

If the map_daemon process is still active, sretarting the map_lpd can be done using the following command:

export MAPPING_PATH=/apps/mapping/conf/mapping.conf
/apps/mapping/bin/map_control -start_lpd
a "scanfolder" robot crashed

When starting a robot, a file with the same name as the robot and a ".ID" extension is created, as well as a file with the process number and and a .pid extension Example:

_apps_mapping_temp_nom_du_robot_map_scanfolder.ID
4656.pid

Were the robot to crash, these files would not be deleted and would prevent the robot from being restarted. To restart the robot, the ".ID" file which corresponds to the crashed robot must then be deleted from the " /apps/mapping/temp " folder

Maintenance

Regular cleaning

The following commands can be set to be executed regularly. Ideally, at least once a day.

Deleting finished and expired jobs
export MAPPING_PATH=/apps/mapping/conf/mapping.conf 
/apps/mapping/bin/map_cron -date
Deleting non finished jobs (no matter their status), after 30 days
export MAPPING_PATH=/apps/mapping/conf/mapping.conf 
/apps/mapping/bin/map_cron -cleanspool -relative_date:30/0/0.  # jour/mois/années
Deleting outdated web session files

context_menu.*, history.*, and *.id files in the /apps/mapping/temp folder

export MAPPING_PATH=/apps/mapping/conf/mapping.conf 
/apps/mapping/bin/map_cron -cleanid
Deleting XPS conversion and Workflow temporary files

These files are located in the /apps/mapping/temp temporary folder. They are normally deleted automatically. In case of problem (crash, forced interruption…), they may remain in the temporary file. It is then not a normal situation.

find /apps/mapping/temp -name "*cri.tmp"     -mtime +2 -exec rm -f {} \;
find /apps/mapping/temp -name "*ttf.tmp"     -mtime +2 -exec rm -f {} \;
find /apps/mapping/temp -name "compress_*"   -mtime +2 -exec rm -f {} \;
find /apps/mapping/temp -name "*.[0-9]"      -mtime +2 -exec rm -f {} \;
find /apps/mapping/temp -name "*.[0-9][0-9]" -mtime +2 -exec rm -f {} \;

Log cleaning and rotation

For practical reasons, the following commands must not scheduled for more than once a day. The aim is to prevent increasing the number of archives created, thus preventing the loss of the logs of the last days. The ideal frequency is 2 times a week, for instance monday night and thursday night, before the night batches are trigerred.

Archiving logs

This command allows you to convert any log file to text format and afterwards, to archive it in a zipped file which shows the date and time.

export MAPPING_PATH=/apps/mapping/conf/mapping.conf 
/apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT

Result:

bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
-rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
-rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
Log rotation of the standard outputs and errors
mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak 
mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
Deleting log archives that are older than 30 jours
find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;
Archiving logs

This command allows you to convert any log file to text format and afterwards, to archive it in a zipped file which shows the date and time. export MAPPING_PATH=/apps/mapping/conf/mapping.conf

/apps/mapping/bin/map_cron -cleanlog -folder:/apps/mapping/spool/logs -format:TXT

Result:

bash-4.2$ ls -lrt /apps/mapping/spool/logs/*.zip
-rw-r--r-- 1 mapadmin staff 254186924 30 oct.  /apps/mapping/spool/logs/2018_10_30_19_51.zip
-rw-r--r-- 1 mapadmin staff  58479965  7 nov.  /apps/mapping/spool/logs/2018_11_07_12_32.zip
Log rotation of the standard outputs and errors
mv  /apps/mapping/temp/stderr.txt  /apps/mapping/temp/stderr.txt.bak 
mv  /apps/mapping/temp/stdout.txt  /apps/mapping/temp/stdout.txt.bak
Deleting log archives that are older than 30 days
find /apps/mapping/spool/logs -name "*.zip" -mtime +30 -exec rm -f {} \;

Sample purge script

#Purge script example
 #!/bin/bash
   echo "------------- Initialization of the parameters necessary for the execution of the script ---------------------"
      echo "Set environment"
         export MAPPING_PATH=/apps/mapping/conf/mapping.conf
      echo "Variable creation containing the path of the Mapping binaries"
         PATH_BIN="/apps/mapping/bin"
      echo "Creation of the other variables used in the script"
         TIMEDATE=`date "+%Y_%m_%d_%H_%M_%S"`
     echo "My variable PATH_BIN :  $PATH_BIN"
     echo "MAPPING_PATH : $MAPPING_PATH"
     echo "Variable creation and population set with the content of the mapping.conf file used by the script"
         PATH_LOG_BACKUP=`$PATH_BIN/map_004 PATH_LOG_BACKUP`
         PATH_BASE_MAPPING=`$PATH_BIN/map_004 PATH_BASE_MAPPING`
     echo "My variable PATH_LOG_BACKUP : $PATH_LOG_BACKUP"
     echo "My variable PATH_BASE_MAPPING : $PATH_BASE_MAPPING"
     echo "My timedate : $TIMEDATE"
  echo "-------------------------- Compression and archiving of the log files/folders ---------------------------------------------"
     echo "Archiivng of the log folder /apps/mapping/spool/logs in to the log backup folder"
         $PATH_BIN/map_cron -cleanlog -folder:$PATH_BASE_MAPPING/mapping_temp/log_temp
     echo "Packaging of the content of the mapping log folder and mapping Log Custom folder"
         tar -cvf $PATH_BASE_MAPPING/mapping_temp/log_temp/Archive_M-Connect_$TIMEDATE.tar $PATH_BASE_MAPPING/spool/mapping_log_custom
         tar -cvf $PATH_BASE_MAPPING/log_archive/log_Archive_$TIMEDATE.tar $PATH_BASE_MAPPING/mapping_temp/log_temp
     echo "Compression of the archive log folder"
         gzip $PATH_BASE_MAPPING/log_archive/log_Archive_$TIMEDATE.tar
  echo "--------------------- Clean of the log folder and mapping log custom folder --------------------------"
     echo "Suppression du contenu du répertoire de log de M-Connect"
         rm -f $PATH_BASE_MAPPING/spool/mapping_log_custom/*
     echo "Suppression du contenu du répertoire de log temporaire"
         rm -f $PATH_BASE_MAPPING/mapping_temp/log_temp/*
     echo "clean of the saved spool files"
         $PATH_BIN/map_cron -cleanspool -state:saved
     echo "Clean of .id , .pid , historique , context_menu files of the mapping /apps/mapping/temp folder"
         $PATH_BIN/map_cron -cleanid
     echo "Clean of the temporally files older than 2 days of the /apps/mapping/temp folder"
         find $PATH_BASE_MAPPING/temp -name "*cri.tmp" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "*ttf.tmp" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "compress_*" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "*.[0-9]" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "*.[0-9][0-9]" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "xps_*" -mtime +2 -exec rm -f {} \;
         find $PATH_BASE_MAPPING/temp -name "tmp_report*" -mtime +2 -exec rm -f {} \;
     echo "Suppression du contenu de plus de 30 jours du répertoire d'archivage /apps/mapping/log_archive"
         find $PATH_BASE_MAPPING/log_archive -name "*" -mtime +30 -exec rm -f {} \;