Перейти к содержанию

NOC 19.2 is Released

In accordance to our Release Policy we're proudly present release 19.2.

19.2 release contains of 337 bugfixes, optimisations and improvements.

Highlights

ConfDB

19.2 introduces new configuration processing framework, the ConfDB. ConfDB is the crystallisation of our over decade experience of configuration processing. Set of carefully designed tools, clean API and flexible query language offer the unified way of configuration parsing, validation, processing and converting. Target area covers data extraction (without complex scripts), config validation, querying and inter-vendor configuration conversions.

We built ConfDB around idea of abstract intermediate config representation, somewhat similar with programming languages' virtual machines, query language with Prolog-like logical programming capabilities, and common pipeline for config parsing.

Key features comparison against existing config processing models:

AreaOld ModelConfDB
Configuration fetchingVia get_config scriptget_config script, external storage download
Configuration storingGridVCSGridVCS with compression
Configuration parsingpyparsing parsers2-stage, tokenization and normalization
Parser complexityHightokenizers are reused between profiles, normalizers - easy
Parser performanceLowHigh
Config modelPlain list of predefined factstree-like structure
Custom factsAlmost impossibleEasy
Query language syntaxCLIPSPython
Query debuggingshellBuilt-in IDE
Discovery integrationNoneConfig-driven discovery checks

19.2 contains ConfDB support for EdgeCore.ES, Eltex.MES, Huawei.VRP, Juniper.JUNOS, MikroTik.RouterOS and Qtech.QSW2800 profiles.

Read more ...

Headless Mode

ConfDB allows the headless mode of operations, where "headless" means no direct access to equipment. Instead, configuration files are downloaded from external storage via FTP, SFTP or S3 API, stored to GridVCS and processed via ConfDB. Discovery can extract capabilities, interfaces and vlans directly from config. IPAM is also integrated with ConfDB, allowing to track vpns, address and prefixes usage.

Headed/Headless mode may be selected on per-device or per-profile basics via appropriate policies.

Headless mode is a valuable tool for due diligences, security audits and forensics.

Config fetch policy

NOC got additional configuration to select, which configuration to prefer - startup or running. Config fetch policy may be applied on per-profile or per-object level.

IPAM

IPAM got various improvements, including: * separate calculation of address and prefix usage * prefix profile's setting to allow reuse of special addresses (network and broadcast)

Ethernet services

IGetInterfaces got the ES address family, allowing to describe complex vlan tag manipulations, often referred as Ethernet Services, Q-in-Q and so on.

GridVCS improvements

19.2 contains simple Mercurial's mpatch implementation to read old deltas. So NOC is no longer depends on Mercurial.

New profiles

19.2 brings support for various IP-cams, including video-specific metrics. Hikvision.DSKV8, Dahua.DH, Axis.VAPIX profiles support lots of IP-cams out-of-the-box.

In addition, GWD.GFA profile has been added.

ProfileChecker

Profile checking engine (discovery's profile check) has been refactored to separate class available both for discovery and for custom scripting.

DNS Zone expiration checks

19.2 restores DNS Zone expiration check, regressed long ago in microservices branch.

API Improvements

Script http client

Scripts' http client has been reworked and got middleware support, allowing various wierd authentication schemes. Integration with web-only managed devices became easier.

nbi telemetry API

NBI service got a telemetry API for metrics pushing from remote agents, making the NOC full-blown telemetry collection and processing system for custom IoT applications.

Code quality

PEP8

Code Quality always has been one of the major priorities. 19.2 contains lots of code cleaning, improving overall code quality and PEP8 compliance.

Tests

Testing infrastructure has been improved and we'd reached 100% test suite passing. Tests passing made the requirement for merge request acceptance.

Python3 compatibility

Python 2.7 will reach end-of-life at 2020-01-01, so we started complex porting project. Additional py3 compatibility linter has been added to the testing lint stage. Python3 compatibility became the requirement for merge request acceptance.

Additional py3 infrastructure for test suite running is ready and we're working on 100% tests passing.

Breaking Changes

IGetConfig

IGetConfig interface got additional policy input parameter. get_config scripts should recognize it and behave accordingly.

Possible values:

  • r - prefer running config
  • s - prefer startup config

Migration

Migration from 19.2 release is straightforward. Just deploy 19.2 or stable-19.2 tags from Tower.