NOC 19.2 is Released¶
In accordance to our Release Policy we're proudly present release 19.2.
19.2 release contains of 337 bugfixes, optimisations and improvements.
Highlights¶
ConfDB¶
19.2 introduces new configuration processing framework, the ConfDB. ConfDB is the crystallisation of our over decade experience of configuration processing. Set of carefully designed tools, clean API and flexible query language offer the unified way of configuration parsing, validation, processing and converting. Target area covers data extraction (without complex scripts), config validation, querying and inter-vendor configuration conversions.
We built ConfDB around idea of abstract intermediate config representation, somewhat similar with programming languages' virtual machines, query language with Prolog-like logical programming capabilities, and common pipeline for config parsing.
Key features comparison against existing config processing models:
| Area | Old Model | ConfDB |
|---|---|---|
| Configuration fetching | Via get_config script | get_config script, external storage download |
| Configuration storing | GridVCS | GridVCS with compression |
| Configuration parsing | pyparsing parsers | 2-stage, tokenization and normalization |
| Parser complexity | High | tokenizers are reused between profiles, normalizers - easy |
| Parser performance | Low | High |
| Config model | Plain list of predefined facts | tree-like structure |
| Custom facts | Almost impossible | Easy |
| Query language syntax | CLIPS | Python |
| Query debugging | shell | Built-in IDE |
| Discovery integration | None | Config-driven discovery checks |
19.2 contains ConfDB support for EdgeCore.ES, Eltex.MES, Huawei.VRP, Juniper.JUNOS, MikroTik.RouterOS and Qtech.QSW2800 profiles.
Headless Mode¶
ConfDB allows the headless mode of operations, where "headless" means no direct access to equipment. Instead, configuration files are downloaded from external storage via FTP, SFTP or S3 API, stored to GridVCS and processed via ConfDB. Discovery can extract capabilities, interfaces and vlans directly from config. IPAM is also integrated with ConfDB, allowing to track vpns, address and prefixes usage.
Headed/Headless mode may be selected on per-device or per-profile basics via appropriate policies.
Headless mode is a valuable tool for due diligences, security audits and forensics.
Config fetch policy¶
NOC got additional configuration to select, which configuration to prefer - startup or running. Config fetch policy may be applied on per-profile or per-object level.
IPAM¶
IPAM got various improvements, including: * separate calculation of address and prefix usage * prefix profile's setting to allow reuse of special addresses (network and broadcast)
Ethernet services¶
IGetInterfaces got the ES address family, allowing to describe complex vlan tag manipulations, often referred as Ethernet Services, Q-in-Q and so on.
GridVCS improvements¶
19.2 contains simple Mercurial's mpatch implementation to read old deltas. So NOC is no longer depends on Mercurial.
New profiles¶
19.2 brings support for various IP-cams, including video-specific metrics. Hikvision.DSKV8, Dahua.DH, Axis.VAPIX profiles support lots of IP-cams out-of-the-box.
In addition, GWD.GFA profile has been added.
ProfileChecker¶
Profile checking engine (discovery's profile check) has been refactored to separate class available both for discovery and for custom scripting.
DNS Zone expiration checks¶
19.2 restores DNS Zone expiration check, regressed long ago in microservices branch.
API Improvements¶
Script http client¶
Scripts' http client has been reworked and got middleware support, allowing various wierd authentication schemes. Integration with web-only managed devices became easier.
nbi telemetry API¶
NBI service got a telemetry API for metrics pushing from remote agents, making the NOC full-blown telemetry collection and processing system for custom IoT applications.
Code quality¶
PEP8¶
Code Quality always has been one of the major priorities. 19.2 contains lots of code cleaning, improving overall code quality and PEP8 compliance.
Tests¶
Testing infrastructure has been improved and we'd reached 100% test suite passing. Tests passing made the requirement for merge request acceptance.
Python3 compatibility¶
Python 2.7 will reach end-of-life at 2020-01-01, so we started complex porting project. Additional py3 compatibility linter has been added to the testing lint stage. Python3 compatibility became the requirement for merge request acceptance.
Additional py3 infrastructure for test suite running is ready and we're working on 100% tests passing.
Breaking Changes¶
IGetConfig¶
IGetConfig interface got additional policy input parameter. get_config scripts should recognize it and behave accordingly.
Possible values:
r- prefer running configs- prefer startup config
Migration¶
Migration from 19.2 release is straightforward. Just deploy 19.2 or stable-19.2 tags from Tower.