Application Configuration
Application configuration including the list of registered framework parsers, publishers and associated properties.
Last Updated: 26 May 2022 • Page Author: Jillur Quddus
Overview
The POB Spring application configuration includes the list of registered framework parsers and publishers executed by the POB data pipeline, along with associated connectivity properties.
Location
The configuration for the POB Spring application context may be found in the pob-configuration Maven module at src/main/resources/application.yml. The complete configuration file is as follows:
Namespaces
frameworks.*
This namespace contains the exhaustive list of procurement frameworks and associated Java-based parser classes that will be executed by the POB data pipeline. If you are extending POB and developing a custom procurement framework parser, then you must add your custom procurement framework to the list in this namespace.
| Property | Description | Example Value |
|---|---|---|
id | An unique string identifier for this procurement framework. | dos |
name | A descriptive name for this procurement framework. | Digital Outcomes and Specialists Framework |
enabled | Whether to enable the parser for this procurement framework during execution of the POB data pipeline. | true |
baseUrl | The base URL to this procurement framework. | https://www.digitalmarketplace.service.gov.uk |
parserClass | The fully qualified class name of the Java-based parser class that will be instantiated and executed during execution of the POB data pipeline. | ai.hyperlearning.pob.data.parsers.dos.DosParser |
filter | Whether to apply a keyword filter to opportunities that have been parsed and extracted from this procurement framework. | true |
keywords | If the keyword filter is enabled, then a space-delimited list of keywords that will be used to filter opportunities. | java python artificial intelligence digital |
properties | An arbitrary list of key-value properties, such as the URL that will be used by the parser to extract procurement opportunities, that will be automatically injected and made available to the Java-based parser class. Note that sensitive properties such as usernames and passwords should be set as externalised variables (for example in HashiCorp Vault or AWS Secrets Manager which are loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml. | opportunitiesUrl: https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities?q=&statusOpenClosed=open |
publishers.*
This namespace contains the exhaustive list of Java-based publisher classes and associated properties that will be executed by the POB data pipeline. If you are extending POB and developing a custom publisher, then you must add your custom publisher to the list in this namespace.
| Property | Description | Example Value |
|---|---|---|
id | An unique string identifier for this publisher. | slack |
enabled | Whether to enable this publisher during execution of the POB data pipeline. | true |
publisherClass | The fully qualified class name of the Java-based publisher class that will be instantiated and executed during execution of the POB data pipeline. | ai.hyperlearning.pob.data.publishers.slack.SlackPublisher |
properties | An arbitrary list of key-value properties, such as webhook URL, that will be automatically injected and made available to the Java-based publisher class. Note that sensitive properties such as webhook URLs, usernames and passwords should be set as externalised variables (for example in HashiCorp Vault or AWS Secrets Manager which are loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml. | webhook: https://hooks.slack.com/services/TSW12345 |
pipelines.main
This namespace contains the configuration for the main end-to-end POB data pipeline which is responsible for executing all procurement framework parsers and publishers registered in application.yml.
| Property | Description | Example Value |
|---|---|---|
enabled | Whether the main POB data pipeline is enabled. | true |
scheduler.enabled | Whether to enable a scheduler that will execute the main POB data pipeline based on a defined Spring CRON schedule. Note that this setting is only applicable when running POB in a self-managed environment. If POB is deployed as an AWS lambda application, then you should configure an AWS EventBridge trigger. If POB is deployed as a Microsoft Azure function app, then the CRON schedule defined in the Azure Handler class will be used instead. | true |
scheduler.cron | If POB is to be deployed to a self-managed environment and the scheduler is enabled, then this setting defines the Spring-based CRON schedule. | 0 0/20 * * * * |
bulkPublicationDelay.enabled | Whether to enable a delay between publishing opportunities in bulk. | true |
bulkPublicationDelay.duration | The duration (in seconds) of the delay between publishing opportunities in bulk. | 10 |
storage.rdbms
This namespace contains the connectivity properties to a relational database management system (RDBMS) that POB will use to act as the master source for all opportunities that have been extracted from the registered procurement frameworks.
| Property | Description | Example Value |
|---|---|---|
driverClassName | The JDBC driver to use when connecting to the RDBMS. | com.mysql.cj.jdbc.Driver |
jdbcUrl | The JDBC connection string that POB will use to connect to the RDBMS. Note that the JDBC URL should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml. | jdbc:mysql://example.com:3306/pob |
username | The username to authenticate the connection to the RDBMS. Note that the username should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml. | pob |
password | The password to authenticate the connection to the RDBMS. Note that the password should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml. | password123 |
POB utilises the Java Spring Framework which provides extensive support for working with SQL databases, including but not limited to HSQLDB, MySQL, MariaDB, PostgreSQL, SQLite and Microsoft SQL Server.
Last updated