Application Configuration
Application configuration including the list of registered framework parsers, publishers and associated properties.
Last Updated: 26 May 2022 • Page Author: Jillur Quddus
Overview
The POB Spring application configuration includes the list of registered framework parsers and publishers executed by the POB data pipeline, along with associated connectivity properties.
Location
The configuration for the POB Spring application context may be found in the pob-configuration Maven module at src/main/resources/application.yml. The complete configuration file is as follows:
Namespaces
frameworks.*
This namespace contains the exhaustive list of procurement frameworks and associated Java-based parser classes that will be executed by the POB data pipeline. If you are extending POB and developing a custom procurement framework parser, then you must add your custom procurement framework to the list in this namespace.
id
An unique string identifier for this procurement framework.
dos
name
A descriptive name for this procurement framework.
Digital Outcomes and Specialists Framework
enabled
Whether to enable the parser for this procurement framework during execution of the POB data pipeline.
true
baseUrl
The base URL to this procurement framework.
https://www.digitalmarketplace.service.gov.uk
parserClass
The fully qualified class name of the Java-based parser class that will be instantiated and executed during execution of the POB data pipeline.
ai.hyperlearning.pob.data.parsers.dos.DosParser
filter
Whether to apply a keyword filter to opportunities that have been parsed and extracted from this procurement framework.
true
keywords
If the keyword filter is enabled, then a space-delimited list of keywords that will be used to filter opportunities.
java python artificial intelligence digital
properties
An arbitrary list of key-value properties, such as the URL that will be used by the parser to extract procurement opportunities, that will be automatically injected and made available to the Java-based parser class. Note that sensitive properties such as usernames and passwords should be set as externalised variables (for example in HashiCorp Vault or AWS Secrets Manager which are loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml.
opportunitiesUrl: https://www.digitalmarketplace.service.gov.uk/digital-outcomes-and-specialists/opportunities?q=&statusOpenClosed=open
publishers.*
This namespace contains the exhaustive list of Java-based publisher classes and associated properties that will be executed by the POB data pipeline. If you are extending POB and developing a custom publisher, then you must add your custom publisher to the list in this namespace.
id
An unique string identifier for this publisher.
slack
enabled
Whether to enable this publisher during execution of the POB data pipeline.
true
publisherClass
The fully qualified class name of the Java-based publisher class that will be instantiated and executed during execution of the POB data pipeline.
ai.hyperlearning.pob.data.publishers.slack.SlackPublisher
properties
An arbitrary list of key-value properties, such as webhook URL, that will be automatically injected and made available to the Java-based publisher class. Note that sensitive properties such as webhook URLs, usernames and passwords should be set as externalised variables (for example in HashiCorp Vault or AWS Secrets Manager which are loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml.
webhook: https://hooks.slack.com/services/TSW12345
pipelines.main
This namespace contains the configuration for the main end-to-end POB data pipeline which is responsible for executing all procurement framework parsers and publishers registered in application.yml.
enabled
Whether the main POB data pipeline is enabled.
true
scheduler.enabled
Whether to enable a scheduler that will execute the main POB data pipeline based on a defined Spring CRON schedule. Note that this setting is only applicable when running POB in a self-managed environment. If POB is deployed as an AWS lambda application, then you should configure an AWS EventBridge trigger. If POB is deployed as a Microsoft Azure function app, then the CRON schedule defined in the Azure Handler class will be used instead.
true
scheduler.cron
If POB is to be deployed to a self-managed environment and the scheduler is enabled, then this setting defines the Spring-based CRON schedule.
0 0/20 * * * *
bulkPublicationDelay.enabled
Whether to enable a delay between publishing opportunities in bulk.
true
bulkPublicationDelay.duration
The duration (in seconds) of the delay between publishing opportunities in bulk.
10
storage.rdbms
This namespace contains the connectivity properties to a relational database management system (RDBMS) that POB will use to act as the master source for all opportunities that have been extracted from the registered procurement frameworks.
driverClassName
The JDBC driver to use when connecting to the RDBMS.
com.mysql.cj.jdbc.Driver
jdbcUrl
The JDBC connection string that POB will use to connect to the RDBMS. Note that the JDBC URL should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml.
jdbc:mysql://example.com:3306/pob
username
The username to authenticate the connection to the RDBMS. Note that the username should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml.
pob
password
The password to authenticate the connection to the RDBMS. Note that the password should be set as an externalised variable (for example in HashiCorp Vault or AWS Secrets Manager which is loaded by the POB bootstrap context) and NOT stored as plaintext in application.yml.
password123
POB utilises the Java Spring Framework which provides extensive support for working with SQL databases, including but not limited to HSQLDB, MySQL, MariaDB, PostgreSQL, SQLite and Microsoft SQL Server.
Last updated