3 PKM server architecture

3.1 Software layers

The PKM server is the backend of the PKM. It is written in javascript language. It manages the data in two persistent storages:

• the MongoDB database server,
• and the server local file system (for storing the Git working trees).

Figure 3 below shows the layered software architecture of the PKM server and its relationship to these two persistent storages:

The bottom layer encompasses the underlying Node.js Javascript run-time, the MongoDB Node.js driver, the MongoDB database server, the local file system, and Git. On top of this, the PKM util layer is a kind of hardware abstraction layer that abstracts file system I/O and simplifies the process of tools execution. The PKM core provides Class PKM, which implements the PKM and is responsible for database accesses using the MongoDB Node.js driver. PKM util and PKM core constitute a Javascript SDK for the PKM whose purpose is mostly to allow third party contributors to extend the PKM. On the highest level, there are two kinds of front-ends:

• the user’s console where the user (or scripts) can interact with the PKM,
• and the user’s interface & tools, which can communicate with the PKM.

These front-ends use two different Application Programming Interfaces:

• The PKM cli provides a command line interface for the PKM whose purposes are essentially doing administrative tasks and debugging the PKM,
• The PKM RESTful HTTP server provides a remote interface over HTTP following the OpenAPI 3 standard whose purpose is fundamentally allowing connecting front-ends, e.g. running within a web browser, or third party tools running on remote servers.

The PKM and parsers source code is available at https://gitlab.ow2.org/decoder/pkm-api/-/blob/master. Because the choosen language for developing the PKM server is Javascript, which is a dynamically typed language, most of the source code has been annotated using the jsdoc markup language (see Appendix A.1) to document the parameters and the return value of each function.

The following subsections presents the MongoDB database and the PKM Application Programming Interfaces. The Git subsystem, which has received a lot of attention, as it is essential software for developers, has Chapter 4 dedicated to it.

3.2 MongoDB database

The MongoDB server can host several databases at a time, and each database can have several users and roles. One special database, named 'admin', defines the MongoDB built-in roles, such as 'root'. A PKM administrator has role 'root' in database 'admin'.

The PKM server maps one PKM project to exactly one MongoDB database. The PKM users, roles, and the list of project names live within a special database called the MongoDB PKM management database (see 'pkm_db' property in File pkm_config.json). The collection in that database for the list of project names is 'Projects'.

Figure 4 below outlines the MongoDB collections in each PKM project:

The Project collection hosts the project metadata (name, members). Each project has a collection for Git working trees metadata, which the PKM server updates after running Git commands. Some collections track the tool executions (invocations and logs). The tool specifications enable GUI front-ends to create well-formed tool invocations (for the Process Engine). The methodology has a collection for the status of each phase of the methodology. The programming artefacts have both collections for files (both executable binaries and source codes), for compile commands (with compilation flags and options), and (after parsing) for source code Abstract Syntax Trees (ASTs), comments, and annotations. The models have also collections for both the UML files, and (after parsing) for UML class diagrams and state machines. The documentation has a collection for the documentation files (e.g. .docx files), for the Abstract Semi-Formal Models (ASFMs), which Doc to ASFM can generate from .docx files, and for the Graphical documentation written in the Graphical Specification Language (GSL). Links discovering tool (Semantic parsing) populates the 2D traceability matrix, while Semantic Role Labeling (SRL) and Named Entity Recognition (NER) tools extract information and synthesize Annotations in a dedicated collection. The TESTAR tool (automated GUI testing) has specialized collections for settings, models, and results. Finally, there are collections for Common Vulnerabilities and Exposures, and the reviews. Note that from the PKM REST API point of view the tool specifications, the TESTAR settings, and the methodology status are also properties of the project, even if in reality they are stored in separate collections. The PKM populates, at project creation, the initial methodology status and a predefined set of tool specifications.

The following subsections detail the collections in a PKM project, which table below summarizes:

Collection	Description
Annotations	Annotations from SRL and NER tools
RawSourcecode	Source code files
sourcecodeC	Source code ASTs for the C language
sourcecodeCPP	Source code ASTs for the C++ language
sourcecodeJava	Source code ASTs for the Java language
annotationsACSL	Source code annotations for the C language
annotationsACSLPP	Source code annotations for the C++ language
annotationsJML	Source code annotations for the Java language
commentsC	Source code comments for the C language
commentsCPP	Source code comments for the C++ language
commentsjava	Source code comments for the Java language
RawDocumentation	Documentation files (e.g. .docx files)
Documentation	Abstract Semi-Formal Models
GraphicalDocumentation	Graphical documentation written in GSL
RawUML	UML files
UMLClasses	UML classes
UMLStateMachines	UML state machines
TESTARSettings	TESTAR settings
TESTARStateModels	TESTAR state models
TESTARTestResults	TESTAR test results
Logs	Execution logs of tools
TraceabilityMatrix	Traceability Matrix
Project	Project metadata
CompileCommands	Compile commands
CVEList	Common Vulnerabilities and Exposures
RawBinaries	Executable binary files
GitWorkingTrees	Git working trees metadata
Tools	Tools specifications
Invocations	Tools invocations
MethodologyStatus	Methodology status
Reviews	Reviews

The JSON schemas for all these collections can be downloaded from the pkm-api gitlab repository.

3.2.1 Project management

In the PKM, a project has a name and members. Each project member has a name and a role in the project. The collection for the Project meta-data is 'Project'. The schema for a project is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-project-schema.json",
  "title": "PKM Project JSON schema",
  "description": "Data model for Project in PKM",
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "project name"
    },
    "members": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string",
            "description": "member name"
          },
          "owner": {
            "type": "boolean",
            "description": "a flag indicating ownership of the project"
          },
          "roles": {
            "type": "array",
            "items": {
              "type": "string",
              "description": "role name",
              "enum": [
                "Owner",
                "Developer",
                "Reviewer",
                "Maintainer"
              ]
            }
          }
        },
        "required": [
          "name"
        ]
      }
    },
    "tools": {
      "type": "array",
      "items": {
        "$ref": "tools.json"
      }
    },
    "methodologyStatus": {
      "type": "array",
      "items": {
        "$ref": "pkm-methodology-status-schema.json"
      }
    },
    "testarSettings": {
      "$ref": "TESTAR_Settings_Schema.json"
    }
  },
  "required": [
    "name"
  ]
}

The PKM users are MongoDB users in the PKM management database. MongoDB natively handles the user name, the password, and the roles.

The PKM defines the following roles:

• Owner
• Developer
• Reviewer
• Maintainer

The PKM uses the ‘customData’ property of MongoDB users for storing some additional information about the user. The schema of a PKM user from the PKM REST API point of view is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-user-schema.json",
  "title": "PKM User JSON schema",
  "description": "Data model for a user in PKM",
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "user's name"
    },
    "password": {
      "type": "string",
      "description": "user's password"
    },
    "first_name": {
      "type": "string",
      "description": "first name"
    },
    "last_name": {
      "type": "string",
      "description": "last name"
    },
    "email": {
      "type": "string",
      "description": "email address"
    },
    "phone": {
      "type": "string",
      "description": "phone number"
    },
    "roles": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "db": {
            "type": "string",
            "description": "database name"
          },
          "role": {
            "type": "string",
            "description": "role name",
            "enum": [
              "Owner",
              "Developer",
              "Reviewer",
              "Maintainer"
            ]
          }
        },
        "required": [
          "db",
          "role"
        ]
      }
    },
    "git_user_credentials": {
      "type": "array",
      "items": {
        "$ref": "pkm-git-user-credential-schema.json"
      }
    }
  },
  "required": [
    "name"
  ]
}

3.2.2 Files

The PKM organizes files per collection depending on the file type. The collections for the PKM files in MongoDB are:

• 'RawSourcecode' for the source code
• 'RawUML' for the UML models
• 'RawDocumentation' for the documentation (e.g. Microsoft Office .docx files)
• 'RawBinaries' for the executable binaries (e.g. Linux ELF files)

From the PKM REST API point of view, these file collections are shown either as four independent endpoints, or as a one unified endpoint for any file. For the later endpoint, the PKM guesses the actual underlying collection by applying some rules about the filename extension and the MIME type. The PKM configuration file pkm_config.json allows extending these rules through the file_types property, see Section 2.1.

The schema below for the PKM files in MongoDB can be downloaded from the pkm-api gitlab repository:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-db-file-schema.json",
  "title": "PKM File JSON schema",
  "description": "Data model for a File in PKM",
  "type": "object",
  "properties": {
    "filename": {
      "type": "string",
      "description": "The relative path (relative to a virtual root directory for the project) of the file (POSIX naming scheme)"
    },
    "filecontent": {
      "type": "string",
      "description": "The content of the file. The original file content shall be encoded as 'base64' (server accepts RFC 4648 §4/§5, and exports RFC 4648 §4) when 'format' field value is 'binary'"
    },
    "fileType": {
      "type": "string",
      "description": "The type of the original file (e.g. Code, Annotation, Comment, or Diagram). When no type is specified, a type may be assigned based on MIME type or filename extension"
    },
    "fileMimeType": {
      "type": "string",
      "description": "The MIME type of the original file (e.g. 'plain/text', 'text/markdown' or 'application/vnd.openxmlformats-officedocument.wordprocessingml.document')"
    },
    "fileFormat": {
      "type": "string",
      "description": "The format of content, either 'text' or 'binary'"
    },
    "fileEncoding": {
      "type": "string",
      "description": "The text encoding of the original file if the original file is a text according to 'fileFormat' field value"
    },
    "gitWorkingTree": {
      "type": "string",
      "description": "The Git working tree path from which the file originates"
    },
    "gitDirty": {
      "type": "boolean",
      "description": "A dirty flag intended for the PKM Git service to indicate that a PKM File has been modified since the last synchronization with the Git working tree regardeless the file is versioned or not"
    },
    "gitUnmerged": {
      "type": "boolean",
      "description": "A flag intended for the user provided by the PKM Git service to indicate that a PKM File is unmerged (merge conflict ?) since the last synchronization with the Git working tree"
    }
  },
  "required": [
    "filename"
  ]
}

It is important to note that a PKM file path is always relative to the virtual root directory of the PKM project.

3.2.3 Source Code

3.2.3.1 Abstract Syntax Trees (ASTs)

The collections for the Abstract Syntax Trees are:

• 'sourcecodeC' for C
• 'sourcecodeJava' for Java
• 'sourcecodeCPP' for C++

The sourcecodeC collection schema is derived from the Frama-C CIL AST. Due to its size, here we only show the main structure. The complete schema can be downloaded from the pkm-api gitlab repository. The main structure is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-c-source-code-schema.json",
  "title": "Source code C Schema Version 2.1.0 JSON Schema.",
  "description": "Source code C Schema Version 2.1.0 JSON Schema.",
  "type": "object",
  "properties": {
    "sourceFile": {
      "type": "string"
    },
    "globals": {
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": true,
        "properties": {
          "GType": {
            "$ref": "#/definitions/GType"
          },
          "GCompTag": {
            "$ref": "#/definitions/GCompTag"
          },
          "GGAnnotCompTag": {
            "$ref": "#/definitions/GGAnnotCompTag"
          },
          "GCompTagDecl": {
            "$ref": "#/definitions/GCompTagDecl"
          },
          "GVar": {
            "$ref": "#/definitions/GVar"
          },
          "GVarDecl": {
            "$ref": "#/definitions/GVarDecl"
          },
          "GEnumTag": {
            "$ref": "#/definitions/GEnumTag"
          },
          "GEnumTagDecl": {
            "$ref": "#/definitions/GEnumTagDecl"
          },
          "GFun": {
            "$ref": "#/definitions/GFun"
          },
          "GFunDecl": {
            "$ref": "#/definitions/GFunDecl"
          },
          "GAnnot": {
            "$ref": "#/definitions/GAnnot"
          },
          "GAsm": {
            "$ref": "#/definitions/GAsm"
          },
          "GPragma": {
            "$ref": "#/definitions/GPragma"
          },
          "GText": {
            "$ref": "#/definitions/GText"
          }
        }
      }
    },
    "globinit": {
      "type": "object",
      "properties": {
        "option": {
          "$ref": "#/definitions/option"
        }
      }
    },
    "globinitcalled": {
      "type": "boolean"
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
}

The sourcecodeCPP collection schema is derived from the Clang AST. Again, due to its size, here we only show the main structure. The complete schema can be downloaded from the pkm-api gitlab repository. The main structure is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-cpp-source-code-schema.json",
  "title": "Source Code C++ Schema Version 1.0.0 JSON Schema.",
  "description": "Source Code C++ Schema Version 1.0.0 JSON Schema.",
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "description": "type of document for the GUI, typically 'Code'"
    },
    "sourceFile": {
      "type": "string",
      "description": "related source code file"
    },
    "manifest": {
      "$ref": "#/definitions/Manifest"
    },
    "inner": {
      "$ref": "#/definitions/Inner"
    }
  },
  "required": [
    "sourceFile",
    "manifest"
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

The sourcecodeJava collection schema is derived from the AST of the Eclipse compiler. Again, due to its size, here we only show the main structure. The complete schema can be downloaded from the pkm-api gitlab repository. The main structure is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-java-source-code-schema.json",
  "title": "Java source code metamodel as JSON schema",
  "type": "object",
  "properties": {
    "CompilationUnit": {
      "type": "object",
      "title": "Compilation unit created from a source document",
      "description": "Java compilation unit AST node type. This is the type of the root of an AST. ",
      "properties": {
        "ImportDeclaration": {
          "$ref": "#/definitions/JavaImportDeclaration"
        },
        "TypeDeclaration": {
          "$ref": "#/definitions/JavaTypeDeclaration"
        },
        "PackageDeclaration": {
          "$ref": "#/definitions/JavaPackageDeclaration"
        },
        "EnumDeclaration": {
          "$ref": "#/definitions/JavaEnumDeclaration"
        },
        "AnnotationTypeDeclaration": {
          "$ref": "#/definitions/JavaAnnotationTypeDeclaration"
        },
        "ModuleDeclaration": {
          "$ref": "#/definitions/JavaModuleDeclaration"
        }
      }
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
  "required": [
    "CompilationUnit"
  ]
}

3.2.3.2 Comments

The collections for the comments are:

• 'commentsC' for C
• 'commentsjava' for Java
• 'commentsCPP' for C++

The comments contain informal text that is associated to some Sourcecode global object or statement.

The comments have the following schema for the C language:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-c-comments-schema.json",
  "title": "C Comments Schema Version 2.0.0 JSON Schema.",
  "type": "object",
  "properties": {
    "sourceFile": {
      "type": "string"
    },
    "comments": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "global_kind": {
            "$ref": "#/definitions/global_kind"
          },
          "loc": {
            "$ref": "#/definitions/loc"
          },
          "comments": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        }
      }
    }
  },
  "definitions": {
    "pos": {
      "type": "object",
      "properties": {
        "pos_path": {
          "type": "string",
          "description": "Path where the file is located"
        },
        "pos_lnum": {
          "type": "number"
        },
        "pos_bol": {
          "type": "number"
        },
        "pos_cnum": {
          "type": "number"
        }
      }
    },
    "loc": {
      "type": "object",
      "properties": {
        "pos_start": {
          "$ref": "#/definitions/pos"
        },
        "pos_end": {
          "$ref": "#/definitions/pos"
        }
      }
    },
    "global_kind": {
      "type": "string",
      "enum": [
        "GType",
        "GCompTag",
        "GCompTagDecl",
        "GEnumTag",
        "GEnumTagDecl",
        "GVarDecl",
        "GFunDecl",
        "GVar",
        "GFun",
        "GAsm",
        "GPragma",
        "GText",
        "GAnnot"
      ]
    }
  }
}

The comments have the following schema for the C++ language:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-cpp-comments-schema.json",
  "title": "C++ Comments Schema Version 2.0.2 JSON Schema.",
  "type": "object",
  "properties": {
    "sourceFile": {
      "type": "string"
    },
    "comments": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "loc": {
            "$ref": "#/definitions/CppLoc"
          },
          "comments": {
            "type": "array",
            "items": {
              "$ref": "#/definitions/CppComment"
            }
          },
          "id": {
            "description": "id of the commented entity",
            "type": "number"
          }
        }
      }
    }
  },
  "definitions": {
    "CppComment": {
      "anyOf": [
        {
          "type": "string"
        },
        {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      ]
    },
    "CppPos": {
      "type": "object",
      "properties": {
        "pos_path": {
          "type": "string",
          "description": "Path where the file is located"
        },
        "pos_lnum": {
          "type": "number"
        },
        "pos_bol": {
          "type": "number"
        },
        "pos_cnum": {
          "type": "number"
        }
      }
    },
    "CppLoc": {
      "type": "object",
      "properties": {
        "pos_start": {
          "$ref": "#/definitions/CppPos"
        },
        "pos_end": {
          "$ref": "#/definitions/CppPos"
        }
      }
    }
  }
}

The comments have the following schema for the Java language:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-java-comments-schema.json",
  "title": "Java comment JSON Schema.",
  "description": "Java comment JSON Schema.",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "fileEncoding": {
      "type": "string"
    },
    "comments": {
      "type": "array",
      "items": {
        "$ref": "#/definitions/JavaComment"
      }
    },
    "fileMimeType": {
      "type": "string"
    },
    "sourceFile": {
      "type": "string"
    },
    "fileFormat": {
      "type": "string"
    },
    "type": {
      "type": "string",
      "description": "type of document for the GUI, typically 'Comment'"
    }
  },
  "required": [
    "comments",
    "fileEncoding",
    "fileFormat",
    "fileMimeType",
    "sourceFile"
  ],
  "definitions": {
    "JavaLoc": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "pos_start": {
          "$ref": "#/definitions/JavaPos"
        },
        "pos_end": {
          "$ref": "#/definitions/JavaPos"
        }
      },
      "required": [
        "pos_end",
        "pos_start"
      ],
      "title": "LOC"
    },
    "JavaPos": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "pos_cnum": {
          "type": "integer"
        },
        "pos_lnum": {
          "type": "integer"
        }
      },
      "required": [
        "pos_cnum",
        "pos_lnum"
      ],
      "title": "Pos"
    },
    "JavaComment": {
      "type": "object",
      "additionalProperties": false,
      "properties": {
        "loc": {
          "$ref": "#/definitions/JavaLoc"
        },
        "commentInEnvironment": {
          "type": "string"
        },
        "comments": {
          "type": "array",
          "items": {
            "type": "string"
          }
        }
      },
      "required": [
        "comments",
        "commentInEnvironment",
        "loc"
      ],
      "title": "Comment"
    }
  }
}

3.2.3.3 Annotations

Annotations, which consist of functions behaviors, are not part of the source code AST but are extracted during source code analysis.

They are stored in the following collections:

• 'annotationsACSL' for C
• 'annotationsJML' for Java
• 'annotationsACSLPP' for C++

The complete schema for C can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-acsl-schema.json",
  "title": "ACSL Schema Version 2.0.0 JSON Schema.",
  "description": "ACSL Schema Version 2.0.0 JSON Schema.",
  "type": "object",
  "properties": {
    "sourceFile": {
      "type": "string"
    },
    "annotations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "function_name": {
            "type": "string",
            "description": "Name of the function being analyzed"
          },
          "loc": {
            "$ref": "#/definitions/loc"
          },
          "behaviours": {
            "type": "array",
            "items": {…}
          }
        }
      }
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
}

The complete schema for Java can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/jml-schema.json",
  "description": "Metamodel definition of the JML language Version 2.1.2 Beta JSON Schema",
  "type": "object",
  "additionalProperties": true,
  "properties": {
    "fileEncoding": {
      "type": "string"
    },
    "annotations": {
      "type": "array",
      "items": {
        "$ref": "#/definitions/JmlAnnotation"
      }
    },
    "fileMimeType": {
      "type": "string"
    },
    "sourceFile": {
      "type": "string"
    },
    "fileFormat": {
      "type": "string"
    },
    "type": {
      "type": "string",
      "description": "type of document for the GUI, typically 'Comment'"
    }
  },
  "required": [
    "annotations",
    "fileEncoding",
    "fileFormat",
    "fileMimeType",
    "sourceFile"
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

The schema below for C++ can be downloaded from the pkm-api gitlab repository:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-acslpp-schema.json",
  "title": "ACSL++ Comments Schema Version 1.0.1 JSON Schema.",
  "type": "object",
  "properties": {
    "sourceFile": {
      "type": "string"
    },
    "annotations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "loc": {
            "$ref": "#/definitions/AcslppLoc"
          },
          "annotations": {
            "type": "array",
            "items": {
              "type": "string"
            }
          },
          "id": {
            "description": "id of the annotated entity",
            "type": "number"
          }
        }
      }
    }
  },
  "definitions": {
    "AcslppPos": {
      "type": "object",
      "properties": {
        "pos_path": {
          "type": "string",
          "description": "Path where the file is located"
        },
        "pos_lnum": {
          "type": "number"
        },
        "pos_bol": {
          "type": "number"
        },
        "pos_cnum": {
          "type": "number"
        }
      }
    },
    "AcslppLoc": {
      "type": "object",
      "properties": {
        "pos_start": {
          "$ref": "#/definitions/AcslppPos"
        },
        "pos_end": {
          "$ref": "#/definitions/AcslppPos"
        }
      }
    }
  }
}

3.2.4 UML classes & state machines

The collections for UML are:

• 'UMLClasses' for UML classes
• 'UMLStateMachines' for UML state machines

The complete schema for UML classes can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-uml-class-diagram-schema.json",
  "title": "UML Class Diagram Schema - Version 1.0.0 JSON Schema.",
  "type": "object",
  "properties": {
    "type": {
      "type": "string"
    },
    "name": {
      "type": "string"
    },
    "diagram": {
      "type": "string"
    },
    "modules": {
      "$ref": "#/definitions/UmlModules"
    }
  },
  "required": [
    "type",
    "name",
    "modules"
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

The complete schema for UML state machines can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-uml-state-model-schema.json",
  "title": "UML State Model Schema - Version 1.0.0 JSON Schema.",
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "id": {
      "type": "string"
    },
    "diagram": {
      "type": "string"
    },
    "regions": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "subvertexes": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {…},
              "required": […]
            }
          },
          "name": {
            "type": "string"
          },
          "id": {
            "type": "string"
          },
          "transitions": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {…},
              "required": […]
            }
          }
        },
        "required": [
          "subvertexes",
          "name",
          "id",
          "transitions"
        ]
      }
    }
  },
  "required": [
    "regions",
    "name",
    "id"
  ]
}

3.2.5 Abstract Semi-Formal Model (ASFM) and Graphical documentation (in GSL)

The collection for Abstract Semi-Formal Models is 'Documentation', while the collection for the graphical documentation is 'GraphicalDocumentation'.

The complete schema for ASFM can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-asfm-schema.json",
  "title": "ASFM (Abstract Semi-Formal Model)",
  "description": "ASFM JSON schema",
  "type": "object",
  "required": [
    "name"
  ],
  "properties": {
    "type": {
      "type": "string"
    },
    "sourceFile": {
      "type": "string"
    },
    "name": {
      "type": "string"
    },
    "units": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [
          "name"
        ],
        "properties": {
          "name": {
            "type": "string"
          },
          "id": {
            "type": "integer"
          },
          "classes": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [
                "name"
              ],
              "properties": {
                "name": {…},
                "id": {…},
                "parent": {…},
                "doc": {…},
                "invariants": {…},
                "fields": {…},
                "methods": {…},
                "types": {…},
                "macros": {…},
                "constants": {…}
              }
            }
          }
        }
      }
    }
  }
}

The complete schema for the graphical documentation of a class written in GSL can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-gsl-schema.json",
  "title": "Graphical Specification Language",
  "description": "Graphical Specification Language",
  "type": "object",
  "properties": {
    "class": {
      "type": "string",
      "description": "name of the class to draw"
    },
    "object": {
      "type": "string",
      "description": "object = instance of the class"
    },
    "content": {
      "$ref": "#/definitions/GSLContent"
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
}

3.2.6 Compile commands

The collection for compile commands, which contain the compiler command options and flags for each compilation units, is 'CompileCommands'. The PKM compile commands derives from LLVM compile commands with one exception: the directory and file properties are paths relative to the virtual root directory of the PKM project.

The complete schema for compile commands can be downloaded from the pkm-api gitlab repository. The schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-compile-command-schema.json",
  "title": "PKM Compile command",
  "description": "Derived from Compilation Database Format JSON specification but with different meaning for directory and file properties",
  "oneOf": [
    {
      "$ref": "#/definitions/CompileCommand"
    },
    {
      "$ref": "#/definitions/CompileCommandWithArguments"
    }
  ],
  "definitions": {
    "CompileCommand": {
      "type": "object",
      "properties": {
        "directory": {
          "$ref": "#/definitions/Directory"
        },
        "file": {
          "$ref": "#/definitions/File"
        },
        "command": {
          "$ref": "#/definitions/Command"
        },
        "output": {
          "$ref": "#/definitions/Output"
        }
      },
      "required": [
        "directory",
        "file",
        "command"
      ]
    },
    "CompileCommandWithArguments": {
      "type": "object",
      "properties": {
        "directory": {
          "$ref": "#/definitions/Directory"
        },
        "file": {
          "$ref": "#/definitions/File"
        },
        "arguments": {
          "$ref": "#/definitions/Arguments"
        },
        "output": {
          "$ref": "#/definitions/Output"
        }
      },
      "required": [
        "directory",
        "file",
        "arguments"
      ]
    },
    "Directory": {
      "type": "string",
      "description": "working directory of the compilation relative to the virtual root directory of a project"
    },
    "File": {
      "type": "string",
      "description": "path of main translation unit source processed by this compilation step relative to the virtual root directory of a project"
    },
    "Command": {
      "type": "string",
      "description": "compile command executed; preprocessor include directives are relative to \"directory\" property (mutually exclusive with \"arguments\")"
    },
    "Arguments": {
      "type": "array",
      "items": {
        "type": "string",
        "description": "command line argument; preprocessor include directives are relative to \"directory\" property (mutually exclusive with \"command\")"
      }
    },
    "Output": {
      "type": "string",
      "description": "name of the output created by this compilation step"
    }
  }
}

3.2.7 Common Vulnerabilities and Exposures (CVE)

The collection for CVE list is 'CVEList'.

The complete schema for CVE list entries can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-cve-schema.json",
  "type": "object",
  "properties": {
    "CVE_data_meta": {
      "type": "object",
      "properties": {
        "ASSIGNER": {
          "type": "string"
        },
        "ID": {
          "type": "string"
        },
        "STATE": {
          "type": "string"
        },
        "UPDATED": {
          "type": "string"
        }
      },
      "required": [
        "ASSIGNER",
        "ID",
        "STATE",
        "UPDATED"
      ]
    },
    "cna-container": {
      "type": "object",
      "properties": {
        "affected": {
          "type": "object",
          "properties": {…},
          "required": […]
        },
        "descriptions": {
          "type": "array",
          "items": {…}
        },
        "problemtypes": {
          "type": "object",
          "properties": {…},
          "required": […]
        },
        "references": {
          "type": "array",
          "items": {…}
        }
      },
      "required": [
        "affected",
        "descriptions",
        "problemtypes",
        "references"
      ]
    },
    "data_format": {
      "type": "string"
    },
    "data_type": {
      "type": "string"
    },
    "data_version": {
      "type": "string"
    }
  },
  "required": [
    "CVE_data_meta",
    "data_format",
    "data_type",
    "data_version"
  ]
}

3.2.8 Annotations

The collection for SRL and NER annotations is 'Annotations'.

The complete schema for Annotations can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-annotations-schema.json",
  "title": "PKM Annotations",
  "description": "JSON Schema for PKM Annotations",
  "type": "object",
  "properties": {
    "path": {
      "type": "string",
      "description": "pkm path to query to retrieve json data containing the text"
    },
    "access": {
      "type": "string",
      "description": "json path allowing to retrieve the text to analyze in the retrieved data"
    },
    "srl": {
      "$ref": "#/definitions/SRLResults"
    },
    "ner": {
      "$ref": "#/definitions/NERResults"
    }
  },
  "required": [
    "path",
    "access"
  ],
  "anyOf": [
    {
      "required": [
        "srl"
      ]
    },
    {
      "required": [
        "ner"
      ]
    }
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

3.2.9 Traceability Matrix

The collection for the 2D traceability matrix, which contains discovered links between artefacts, is 'TraceabilityMatrix'.

The schema below for the traceability matrix can be downloaded from the pkm-api gitlab repository:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-traceability-matrix-schema.json",
  "title": "Traceability Matrix Cell",
  "description": "Traceability matrix cell JSON schema",
  "type": "object",
  "properties": {
    "src_path": {
      "type": "string",
      "description": "Url-encoded path in the pkm e.g. (not encoded for readability)  code/c/comments/mydb"
    },
    "src_access": {
      "type": "string",
      "description": "Url-encoded json path (in JMESPath query language) to access the text in json pointed to by the src_path. E.g. (not encoded for readability): '[?global_kind == 'GFun'] | [?loc.pos_start.pos_path == 'examples/vector2.c'].comments | [0]'"
    },
    "tgt_path": {
      "type": "string",
      "description": "Url-encoded path in the pkm e.g. (not encoded for readability)  code/c/comments/mydb"
    },
    "tgt_access": {
      "type": "string",
      "description": "Url-encoded json path (in JMESPath query language) to access the text in json pointed to by the src_path. E.g. (not encoded for readability): '[?global_kind == 'GFun'] | [?loc.pos_start.pos_path == 'examples/vector2.c'].comments | [1]'"
    },
    "trace": {
      "type": "object",
      "properties": {
        "similarity": {
          "type": "number",
          "description": "The similarity between source and target texts"
        },
        "role": {
          "type": "string",
          "description": "The kind of trace link between the source and the target. Can be empty."
        }
      }
    }
  }
}

3.2.10 Logs and reports

The collection for the logs & reports, which contains both the messages that the tools emit and the analysis results of the tools, is 'Logs'.

The complete schema for the logs & reports can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-log-schema.json",
  "title": "Log",
  "description": "Log JSON schema",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "description": "ID of Log (automatically generated when missing)"
    },
    "tool": {
      "type": "string",
      "description": "tool name/tag"
    },
    "nature of report": {
      "type": "string",
      "description": "e.g \"Proof report\", \"Modeling report\", \"Testing report\", \"GUI report\", \"NER report\", \"Summarization report\", etc."
    },
    "start running time": {
      "type": "string",
      "description": "start running time"
    },
    "end running time": {
      "type": "string",
      "description": "end running time"
    },
    "messages": {
      "$ref": "#/definitions/Messages"
    },
    "warnings": {
      "$ref": "#/definitions/Warnings"
    },
    "errors": {
      "$ref": "#/definitions/Errors"
    },
    "status": {
      "type": "boolean",
      "description": "flag of failure of the process: true=not failed"
    },
    "details": {
      "type": "object",
      "description": "e.g. parameters of the tool, analysis reports, etc."
    }
  },
  "required": [
    "tool",
    "nature of report",
    "start running time",
    "end running time",
    "status"
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

Because the logs can be huge (several hundred of megabytes), the logs in the database may have 'messages', 'warnings', 'errors', and 'details' properties serialized as JSON in a 'content' property.

3.2.11 TESTAR

The collections for TESTAR are:

• 'TESTARSettings'
• 'TESTARStateModels'
• 'TESTARTestResults'

The complete schema for TESTAR settings can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/TESTAR_Settings_Schema.json",
  "title": "TESTAR settings",
  "description": "TESTAR settings",
  "type": "object",
  "properties": {
  },
  "definitions": {
    //Contains all the input definitions
  }
}

Note that TESTAR settings are currently a free-form object, which can be extended later as needed.

The complete schema for TESTAR state models can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "definitions": {
    //Contains all the input definitions
  }
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/TESTAR_StateModel_Schema.json",
  "type": "object",
  "title": "The Root Schema",
  "required": [
    "timestamp",
    "url",
    "sut",
    "tool",
    "stateModelDataStore",
    "stateModelDataStoreType",
    "stateModelDataStoreServer",
    "stateModelDataStoreDirectory",
    "stateModelDataStoreDB",
    "stateModelDataStoreUser",
    "stateModelDataStorePassword",
    "stateModelIdentifier",
    "stateModelAppName",
    "stateModelAppVersion",
    "stateModelDifference",
    "abstractionId",
    "deterministic",
    "unvisitedAbstractActions",
    "numberAbstractStates",
    "numberAbstractActions",
    "numberConcreteStates",
    "numberConcreteActions",
    "storeWidgets",
    "numberWidgets",
    "numberTestSequences",
    "testSequences"
  ],
  "properties": {
    "timestamp": {…},
    "url": {…},
    "sut": {…},
    "tool": {…},
    "stateModelDataStore": {…},
    "stateModelDataStoreType": {…},
    "stateModelDataStoreServer": {…},
    "stateModelDataStoreDirectory": {…},
    "stateModelDataStoreDB": {…},
    "stateModelDataStoreUser": {…},
    "stateModelDataStorePassword": {…},
    "stateModelIdentifier": {…},
    "stateModelAppName": {…},
    "stateModelAppVersion": {…},
    "stateModelDifference": {…},
    "abstractionId": {…},
    "deterministic": {…},
    "unvisitedAbstractActions": {…},
    "numberAbstractStates": {…},
    "numberAbstractActions": {…},
    "numberConcreteStates": {…},
    "numberConcreteActions": {…},
    "storeWidgets": {…},
    "numberWidgets": {…},
    "numberTestSequences": {…},
    "testSequences": {…}
  }
}

The complete schema for TESTAR state results can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "definitions": {
    //Contains all the input definitions
  }
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/TESTAR_TestResults_Schema.json",
  "type": "object",
  "title": "The Root Schema",
  "required": [
    "timestamp",
    "url",
    "sequencesResult",
    "htmlsResult",
    "logsResult",
    "sequencesVerdicts",
    "sut",
    "tool",
    "settings"
  ],
  "properties": {
    "timestamp": {…},
    "url": {…},
    "sequencesResult": {…},
    "htmlsResult": {…},
    "logsResult": {…},
    "sequencesVerdicts": {…},
    "sut": {…},
    "tool": {…},
    "settings": {…}
  }
}

3.2.12 Reviews

The collection for the reviews is 'Reviews'.

The schema below for the reviews can be downloaded from the pkm-api gitlab repository:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-review-schema.json",
  "title": "PKM review schema",
  "description": "PKM review",
  "type": "object",
  "properties": {
    "reviewID": {
      "type": "string"
    },
    "reviewTitle": {
      "type": "string"
    },
    "reviewStatus": {
      "type": "string",
      "enum": [
        "OPEN",
        "CLOSED"
      ]
    },
    "reviewOpenDate": {
      "type": "string"
    },
    "reviewCompletedDate": {
      "type": "string"
    },
    "reviewAuthor": {
      "type": "string"
    },
    "reviewComments": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "comment": {
            "type": "string"
          },
          "author": {
            "type": "string"
          },
          "datetime": {
            "type": "string"
          }
        },
        "required": [
          "comment",
          "author",
          "datetime"
        ]
      }
    },
    "reviewArtifacts": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "path": {
            "type": "string"
          },
          "type": {
            "type": "string",
            "enum": [
              "Diagram",
              "Code",
              "Document",
              "Doc",
              "Annotation",
              "Log",
              "Comment",
              "TESTAR_State_Model",
              "UML Model",
              "NER",
              "SRL"
            ]
          }
        },
        "required": [
          "path",
          "type"
        ]
      }
    }
  },
  "required": [
    "reviewTitle",
    "reviewOpenDate",
    "reviewAuthor",
    "reviewStatus"
  ]
}

3.2.13 Git support

The collection for the Git working trees metadata, which acts as cache to avoid repeatedly scanning the actual Git working trees on the local file system, is 'GitWorkingTrees'.

The schema below for the Git working trees metadata can be downloaded from the pkm-api gitlab repository:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-git-working-tree-schema.json",
  "title": "PKM Git Working Tree JSON schema",
  "description": "Data model for a Git Working Tree in PKM",
  "type": "object",
  "properties": {
    "directory": {
      "type": "string",
      "description": "Directory of the (main) Git working tree"
    },
    "git_directory": {
      "type": "string",
      "description": "Git directory"
    },
    "git_branch": {
      "type": "string",
      "description": "Tracked Git branch"
    },
    "git_commit_id": {
      "type": "string",
      "description": "SHA1 ID of current commit"
    },
    "git_config": {
      "type": "object",
      "description": "Git configuration of the Git directory (usually .git/config) associated to a Git working tree"
    },
    "linked": {
      "type": "array",
      "description": "Linked Git working trees created with 'git worktree add'",
      "items": {
        "type": "object",
        "properties": {
          "main": {
            "type": "string",
            "description": "Directory of the main Git working tree"
          },
          "directory": {
            "type": "string",
            "description": "Directory of the linked Git working tree"
          },
          "git_directory": {
            "type": "string",
            "description": "Git directory"
          },
          "git_branch": {
            "type": "string",
            "description": "Tracked Git branch"
          },
          "git_commit_id": {
            "type": "string",
            "description": "SHA1 ID of current commit"
          },
          "git_config": {
            "type": "object",
            "description": "Git configuration of the Git directory (usually .git/config) associated to a linked Git working tree"
          }
        }
      }
    }
  },
  "required": [
    "git_working_tree",
    "git_branch",
    "git_commit_id"
  ]
}

3.2.14 Process Engine

The collections for the process engine, which orchestrates the execution of tools according to the methodology phases, are:

• 'Tools'
• 'Invocations'
• 'MethodologyStatus'

The complete schema for Tools can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/tools.json",
  "title": "Root",
  "type": "object",
  "required": [
    "toolID",
    "toolName",
    "server",
    "phases",
    "tasks",
    "endpoint"
  ],
  "properties": {
    "toolID": {
      "type": "string"
    },
    "minutesToStuck": {
      "type": "integer"
    },
    "toolName": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "server": {
      "type": "string"
    },
    "phases": {
      "title": "Phases",
      "type": "array",
      "default": […],
      "items": {…}
    },
    "tasks": {
      "title": "Tasks",
      "type": "array",
      "default": […],
      "items": {…}
    },
    "endpoint": {
      "$ref": "#/definitions/ToolEndpoint"
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
}

The complete schema for Invocations can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/invocations.json",
  "type": "object",
  "required": [
    "invocationID",
    "tool",
    "user",
    "timestampRequest",
    "invocationStatus",
    "invocationConfiguration"
  ],
  "properties": {
    "invocationID": {
      "type": "string"
    },
    "tool": {
      "type": "string"
    },
    "invocationConfiguration": {
      "type": "object"
    },
    "user": {
      "type": "string"
    },
    "timestampRequest": {
      "type": "string"
    },
    "timestampStart": {
      "type": "string"
    },
    "timestampCompleted": {
      "type": "string"
    },
    "invocationStatus": {
      "type": "string",
      "enum": […]
    },
    "invocationResults": {
      "type": "array",
      "default": […],
      "items": {…}
    }
  },
  "definitions": {
    //Contains all the input definitions
  }
}

A methodology has several phases, and its global status consists in the status of each phase of the methodology. The complete schema for the phase status of the methodology can be downloaded from the pkm-api gitlab repository. The main structure of the schema is the following:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://gitlab.ow2.org/decoder/pkm-api/-/tree/master/api/pkm-methodology-status-schema.json",
  "title": "Phase",
  "description": "PKM Methodology phase status",
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "phaseNumber": {
      "type": "integer"
    },
    "name": {
      "type": "string"
    },
    "id": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "completed": {
      "type": "boolean"
    },
    "tasks": {
      "type": "array",
      "items": {…}
    }
  },
  "required": [
    "completed",
    "description",
    "id",
    "name",
    "phaseNumber",
    "tasks"
  ],
  "definitions": {
    //Contains all the input definitions
  }
}

3.3 Application Programming Interfaces

3.3.1 Javascript SDK

The development kit is written in Javascript and intended to run with Node.js anywhere (either client side, or server side). It is composed of two parts with clear roles:

• PKM core: it is responsible of database accesses using the MongoDB Node.js driver
• PKM util: it is about abstracting the host file system and providing helper functions to invoke some of the DECODER Tool-chain tools using their command line interface (Frama-C, Frama-Clang, doc_to_asfm, and excavator) and also git.

For a detailed documentation of the Javascript SDK (generated with jsdoc), see Appendix A.1.

In PKM core:

• Class PKM provides methods for:
  • Session management i.e. user’s login and logout,
  • Document management i.e. document insertion, update and deletion,
  • Document querying, i.e. getting documents which match a query,
  • Invalidating dependent documents, e.g. invalidating Abstract syntax trees related to a source code file when that source code file has changed,
  • Validating documents, i.e. checking documents against schemas before inserting or updating the documents in the MongoDB database,
• Class User represents a user,
• Class UserRole represents a user’s role in a project,
• Class GitUserCredential represents the credential for accessing Git remote servers.

In PKM util:

• Class FileSystem provides methods for file system I/O,
• Class File abstracts a file,
• Functions frama_c, frama_clang, doc_to_asfm, excavator and git use the command line interface of the respective tools that run within a working directory managed with Class FileSystem.

The source code of each part of the Javascript SDK is on the pkm-api gitlab repository respectively in directory core and util.

Most of functions and methods use an asynchronous control flow pattern using Javascript Promises. Concretely that means that all API functions and methods are non-blocking and returns immediately an instance of Class Promise, before actual work is finished. The Javascript runtime triggers callbacks once the promise resolves. Code below shows the caveat when coding with promises:

var expected_result;
pkm.foobar().then((result) =>
{
    // (2) code within this scope will unlikely execute before code below (1)
    console.log('result:', result);
    expected_result = result;
}).catch((error) =>
{
    console.error('error:', error);
    process.exit(1);
}
// (1) code below will likely execute before code below (2)
console.log('expected_result:', expected_result); // ⚠️ content of expected_result is likely to be undefined
                                                  //    because not yet assigned
...

Developers not used to using Javascript promises who would like to extend the PKM server can find more information about Javascript promises by consulting the resources below:

• MDN Web Docs: Promise
• Javascript Tutorial: The Definitive Guide to the JavaScript Promises

3.3.1.1 Session management

The PKM session management uses a token bearer authentication scheme. PKM.login, PKM.access and PKM.logout are static methods responsible of user’s login, access control during a user’s session and user’s logout respectively. On successful user’s login, PKM.login returned promise generates a key (the token) that identifies a logged in user, and then create a new instance of Class PKM bound to that key. The PKM.login promise resolution result is the newly created instance of Class PKM. The key and the newly created instance of Class PKM are valid until user’s session expires. Depending on the PKM configuration, the token can expire after an amount of time (after last login or access), see session_timeout option in the PKM configuration file.

Below is an example for login:

const PKM = require('./core/pkm');
const config = {
    db_host : 'server.world.com:27017', // Database server URI
    auth_db : 'users',                  // Authentication database
    debug : false                       // Flag to enable/disable debugging message of the PKM API
};
// Connecting to the database server
PKM.login('garfield', 'password', config).then((pkm) =>
{
    // pkm is an instance of Class PKM which has methods to work with PKM documents
    ...
    // When finished, closing the connection with the database server
    pkm.close();
}).catch((err) =>
{
    console.error(err);
    process.exit(1);
});

When the PKM is running server side and a user is willing to access the PKM through the Internet using a web browser, he needs to authenticate once and then can request for PKM services many times before session expires. To identify an authenticated user while maintaining security, PKM relies on a key (generated by the PKM server), whose validity lasts for a certain amount of time.

Server side code for user’s authentication looks like below:

const PKM = require('./core/pkm');
const config = {
    db_host : 'server.world.com:27017', // Database server URI
    auth_db : 'users',                  // Authentication database
    session_timeout : '1 h',            // Session will expire after 1 hour
    debug : false                       // Flag to enable/disable debugging message of the PKM API
};
// Connecting to the database server
PKM.login('garfield', 'password', config).then((pkm) =>
{
    // Send the key to the user interface
    send_key_to_client(pkm_docker.key);

    // Connection is automatically acquired after login so that it cannot get closed when session expires.
    // When finished, do not close the connection with the database server
    // but instead mark it ready for closing when user session expires.
    pkm.release();

}).catch((err) =>
{
    console.error(err);
    process.exit(1);
});

Client side code looks like below:

send_login_to_server('garfield', 'password');
var key = receive_key_from_server();
sessionStorage.setItem('pkm-key', key); <-- keep key on client side session storage

Server side code for accessing PKM services after a successful login looks like below:

const PKM = require('./core/pkm');
PKM.access(key).then((pkm) =>
{
    // pkm is an instance of Class PKM which has methods to work with PKM documents
    ...
    // Connection is automatically acquired after resolution of access promise so that it cannot get closed when session expires.
    // When finished, do not close the connection with the database server
    // but instead mark it ready for closing when user session expires.
    pkm.release();

}).catch((err) =>
{
    console.error(err);
    process.exit(1);
});

Server side code for user’s logout looks like below:

const PKM = require('./core/pkm');
PKM.logout(key).then((pkm) =>
{
    // connection is closed and session has expired
}).catch((err) =>
{
    console.error(err);
    process.exit(1);
});

3.3.1.2 Document management

Class PKM provides developer with a mechanism in static Method PKM.load_global_config to define new collections in the PKM databases for new types of document. The schemas of these new types of documents can be specified not only in this method but also in PKM configuration File pkm_config.json.

Class PKM provides developer with some generic methods named insert_documents, update_documents, insert_update_documents, and delete_documents to manage these new types of document:

• insert_documents(dbName, collection_name, documents, signature, options)
• update_documents(dbName, collection_name, documents, signature, options)
• insert_update_documents(dbName, collection_name, documents, update, signature, options)
• delete_documents(dbName, collection_name, query = {})

Class PKM also provides developer with insert_documents_spanned, update_documents_spanned, insert_update_documents_spanned, and delete_documents_spanned to manage documents spanned over multiple collections:

• insert_documents_spanned(dbName, collection_names, documents, dispatch, signature)
• update_documents_spanned(dbName, collection_names, documents, dispatch, signature)
• insert_update_documents_spanned(dbName, collection_names, documents, update, dispatch, signature)

There are also some specialized but still quite generic methods named insert_update_files, insert_update_files_spanned and delete_files_spanned to manage files:

• insert_update_files(dbName, collection_name, files, update, options = {})
• insert_update_files_spanned(dbName, collection_names, files, update, dispatch, options = {})
• delete_files_spanned(dbName, collection_names, query = {})

Class PKM provides developer with many specialized methods that rely on these generic methods to manage all types of documents and files that PKM supports.

3.3.1.3 Document querying

Class PKM provides developer with some generic methods named get_documents and get_documents_spanned to query a document in collections:

• get_documents(dbName, collection_name, query = {}, options = {})
• get_documents_spanned(dbName, collection_names, query = {}, options = {})

There are also some specialized but still quite generic methods named get_files, and get_files_spanned to query files:

• get_files(dbName, collection_name, query = {}, options = {})
• get_files_spanned(dbName, collection_names, query = {}, options = {})

Class PKM provides developer with many specialized methods that rely on these generic methods to query all type of documents and files that PKM supports.

3.3.1.4 Dependent document invalidation

The dependent documents related to a file, such as the source code ASTs, comments and annotations documents, get automatically invalidated when a new file is inserted or the content of a file has changed.

This applies to document in the following collections:

• 'sourcecodeC'
• 'annotationsACSL'
• 'commentsC'
• 'sourcecodeCPP'
• 'annotationsACSLPP'
• 'commentsCPP'
• 'sourcecodeJava'
• 'annotationsJML'
• 'commentsjava'

3.3.1.5 Document validation

Class PKM provides developer with some methods to validate a document against a schema before inserting that document into the database or updating an existing document in the database. Note that the generic document management functions of Class PKM can automatically validate documents against the defined schemas for each document collection.

3.3.1.6 File system I/O

Class FileSystem provides an abstraction layer of the host file system. An instance of that class is geared to a directory (either permanent or temporary) on the host file system. This class provides basic I/O function, e.g. readFile and writeFile:

• constructor(root_directory, options)
• get_abs_path(rel_path)
• make_abs_path(_path)
• make_rel_path(abs_path)
• check_abs_file_path(abs_host_file_path)
• check_abs_directory_path(abs_host_dir_path)
• readFile(host_file_path, options)
• writeFile(file, options)
• stat(host_file_path, options)
• access(host_file_path, options)
• unlink(host_file_path)
• mkdir_p(host_dir_path)
• rmdir_r(host_dir_path)
• make_root_directory()
• remove_root_directory()
• make_temporary_root_directory()
• remove_temporary_root_directory()
• fix_path(abs_path)
• find(starting_point, options)

3.3.1.7 Tools invocation

Some functions are provided to simplify the process of executing tools. These functions are the followings:

• frama_c(file_system, compile_commands, mode, options)
• frama_clang(file_system, compile_commands, options)
• doc_to_asfm(file_system, file_path, options)
• asfm_to_doc(asfm_obj, options)
• git(file_system, git_commands, options)
• excavator(file_system, binary, options)

3.3.2 Command Line Interface (CLI)

The PKM command line interface provides administrator and developer with commands for administrative tasks and debugging the PKM. For a detailed documentation of the command line interface, see sibling document titled “Open source client-side software”, Appendix A.2.

3.3.3 REST API

The PKM REST API provides access to the PKM over HTTP/HTTPS. The PKM has an OpenAPI 3 specification available at https://gitlab.ow2.org/decoder/pkm-api/-/blob/master/api/pkm-openapi.yaml, which enables to automatically generate the SDK for many programming languages, see Appendix A.2. Appendix A.2 contains detailed explanations about implementation design of the REST server, which provides front-ends and third party tools developers with the PKM REST API.

D1.5 Open source PKM server-side software - 03/11/22