Перейти к содержанию

NOC 19.4

On this page

In accordance to our Release Policy <releases-policy> we're proudly present release 19.4.

19.4 release contains 386 bugfixes, optimisations and improvements.


Python3 Compatibility Statement

Python3 compatibility was one of primal focuses during 19.4 release cycle. We made lots of code reviews and cleanups to improve py3 compatibility. NOC 19.4 is able to perform test suite both on Python 2.7 and Python 3.6 and most of functions work on py3, though some glitches still remain. So we continue routine work on reaching full py3 compatibility.

We expect that:

  • NOC 20.1 will have full py3 compatibility and will allow to choose between py2 and py3 during deploy.
  • NOC 20.2 will by py3-only release with all py2-compatibility stuff removed.
  • NOC 20.3 will use py3 features like async/await and will switch from tornado to FastAPI.

Extended MAC discovery

Discovery got extended MAC (xmac) discovery method, showing good results in linking dumb terminal equipment like CPE's, IP Phones, IP cams, which have no other reliable discovery methods.

See #1071 for details.

DataStream improvements

IPAM got VRF<api-datastream-vrf>, Prefix<api-datastream-prefix> and Address<api-datastream-address> streams allowing to expose all IPAM data to external systems.

managedobject DataStream<api-datastream-managedobject> got additional tags attribute exposing ManagedObjects's tags. Tags allow to pass unstructured metadata to external system.

managedobject DataStream<api-datastream-managedobject> also exposes comprehensive VPN information.

nbi improvements

NBI got 3 additional groups of API:

  • objectstatus<api-nbi-objectstatus> for online Managed Object status requesting
  • config<api-nbi-config> and configrevisions<api-nbi-configrevisions> for accessing Managed Object's config
  • path<api-nbi-path> - the swiss-army knife for network topology paths calculation

time_delta in metrics

Discovery writes time elapsed from previous check to time_delta field, allowing to store metrics deltas instead of speed, allowing to perform tasks like traffic calculation and robust error detection.

Objects can be searched and centered on network map which can be helpful on large maps.

Compare Configs

Configs from different managed objects can be compared via UI

BI improvements

Amount of reboots, outages, STP topology changes are added to Managed Objects' BI statistics allowing additional problem drill-down vectors.

ICQ sender

Additional icqsender service allows to push notifications via ICQ messenger.

Development Process Changes

py3 tests

Python3 tests are mandatory now and all test suite must pass python3 testings stages as well.

Mypy integration

NOC adopts Mypy. Though mypy tests are still advisory developers are urged to add type annotations to new and to the refactored code.

Code Formatting

NOC adopts Prettier for JS, JSON, CSS and YAML code formatting. CI pipeline checks code formatting of changed files. Any misformatting considered the error and CI pipeline fails at the lint stage. We recommend to add Prettier formatting to git's pre-commit hook or to the IDE's on-save hook.

Docker container is also available. Use:

docker run --rm \
    -w /src \
    -v $PWD:/src \
    registry.getnoc.com/infrastructure/prettier:master \
    /usr/local/bin/prettier --config=.prettierrc --write <file name>

to format file

New features

MR2234network segment: search & select object
MR2374Took default user password and email from env on initial migrations
MR2532nbi: config and configrevisions API
MR2613metrics: Collect time_delta parameter
MR2655DataStream: ManagedObject tags
MR2671Config diff from two objects' revisions
MR2714noc/noc#1126 Add n_reboots, availability, total_unavailability, n_outages,...
MR2727Path API
MR2764DataStream: Expose VPN information to managedobject datastream
MR2772Add ICQ sender notification service
MR2862Extended MAC discovery
MR2881datastream: IPAM integration
MR2886nbi: objectstatus API


MR1787Add full address on map
MR1990Replace pylibssh2 with ssh2-python
MR2408docs: Fix profile anchors
MR2415ConfDB: Syntax documentation
MR2492Add discovery status classificator to report object stat.
MR2504Check for nbi MR label
MR2506cmtext height 100%
MR2508Filter enable_discovery False from Report Profile Check Summary.
MR2533login: Log authentication failures
MR2548Change to Edit in ModelApplication.js title
MR2562translate syslog
MR2567Set secure and httponly cookie
MR2571prepare for build
MR2574Prettier code formatting check
MR2575pynsq 0.8.3
MR2576Django 1.11.24
MR2577pyfilesystem2 2.4.10
MR2591NSQ http publisher
MR2594escalation: Configurable group tt promotion
MR2598Add test_prefixlist.py
MR2602Add IPv4 address format check to ManagedObject etl loader.
MR2609Optimize uplink calculations for complex topologies
MR2614#955 sa.managedobject: Group editing of description
MR2625ClickHouse: Insert into JSONEachRow format
MR2627python-consul 1.1.0
MR2636Add ignore_empty_cli_response to script cli method.
MR2638Add loader to GeocoderCache.
MR2656sa.managedobject: Additional access checks in the config preview
MR2669Cleanup CH sharding code
MR2673JSONL format for BI ETL
MR2693Add ObjectProfile field to Alarm and Reboot BI models.
MR2712Add arguments to BI extract method.
MR2717Fix the name of the standard for Gigabit Ethernet.
MR2718pytest 4.6.6, Coverage 4.5.4, removed unused option ./noc test --test-report
MR2722Extend logging on MIB import.
MR2730sa.managedobjectprofile, inv.interfaceprofile: Case-insensitive substring search
MR2742maintenance networksegment type changed from lookup to tree
MR2754MO profile, numberfield minValue set
MR2762ConfDB: Data Types
MR2763ConfDB: MPLS protocol
MR2791RepoPreview style fixed
MR2799Add ICQ sender service docs
MR2813modelfilter.List remove padding: 10
MR2823Add segment mac to discovery command.
MR2824Add forwarding_instance index to subinterface model.
MR2825Add config.version_format for noc version customization.
MR2829Extend logging when ConsulDCS lost session.
MR2846Fix icqsender doc
MR2855Add SNMP encode_oid function speedup.
MR2871Add ObjectProfile fieild to ManagedObject BI model.
MR2889fix duration for active alarms
MR2892CI: Run python3 tests automatically
MR2906Speedup ReportAvailability.
MR2908Add url storage path to beef command
MR2921noc/noc#1159 Add eof_mark param and extend logging on digestauth.
MR2924noc/noc#1157 Add filter_vendor, filter_platform, filter_version to ManagedObjectSelector application.
MR2929Add new get inventory2
MR2934Add description param to beef collect command.
MR2938Add ad-hoc beef options to script
MR2940PyFilesystem2 2.4.11
MR2943noc/noc#1104 Add static MAC to mac discovery.
MR2947Update HP.ProCurve profile
MR2958noc/noc#1158 Disconnect connections on asset discovery.
MR2959py3: Set py3 tests as mandatory
MR2962Replace interface profile on nri_service discovery when change settings.
MR2964noc/noc#1088 Add ifindex and mac to confdb interface meta.
MR2969Add to beef command allow use local specs.
MR2972Add ERR_SNMP_BAD_COMMUNITY error when credentials without SNMP.
MR2979noc/noc#1098 Remove read() from read json file in inject-event command.
MR2997Speedup DiscoveryTopology Problem report.
MR3001Use sys.platform unstead os.uname().


MR2281Fix HP.ProCurve.get_lldp_neighbors script
MR2446Add DISABLE_SERVER_SIDE_CURSOR params to connect django db.
MR2463form objectvalidationpolicy fixed
MR2464Fix KB index page trace when empty user history.
MR2481Remove newline from version info
MR2486Fix interface-profile command trace when connect.
MR2490Fix ddash when no metric on interface profile
MR2501Fix matcher parsing
MR2503Ensure User.last_login is nullable
MR2535inv.map right width increase
MR2536Fix on_init attribute on administrative_domain model.
MR2545Use ReportModelFilter for ReportDiscoveryLinks.
MR2547Change user_permissions field to permissions (fix create user).
MR2549Add ch_escape to managedObject BI extractor.
MR2558noc/noc#1095 Fix mongo connect on commands job, run, wipe.
MR2559Fix DiscoveryResult reportdatasource high memory consumption.
MR2560Register unknown part_no use first vendor_code
MR2568Add object_profile field to managed_object iter_datastream.
MR2570translate Add Insert for listformfield and gridfield fixed
MR2572Add use_mongo to mrt services.
MR2587#1105 Fix apply fix_object_uplinks and fix_alarm_managedobjectprofile
MR2595Fix ReportObjectDetail append row.
MR2600#1100 Fix FirmwarePolicy.get_recommended_version()
MR2612collections: Ignore unknown fields
MR2618Update commands/whois.py
MR2621Fix mongo connect on whois command.
MR2622Fix clear unknown model when create.
MR2629Add UUID to JSON Unique field on Inventory models.
MR2634#1112 fix csv-import/export command
MR2637Fix interface_flap param on ReportMetric.
MR2660Fix broke --clean argument on events command
MR2666Fix 'code' object is not callable on metric shard function.
MR2696Fix syslogcollector.register_message typo.
MR2699requirements tablesorter & table2csv added
MR2700#1122 Split too large published messages into parts
MR2710Fix CSV Import
MR2713noc/noc#1122 Workaround for stuck metrics queue when message very large.
MR2716Fix _iter_metrics_raw_chunks chunk limit config.
MR2747Handle NSQ publisher failures
MR2765Fix escalation Stop Processing
MR2766Set availability ManagedObject BI extractor to 100%.
MR2773Chrome ignores autocomplete='off' - fixed
MR2798Catch SyntaxError trace when managedObject card open.
MR2800NBI objectmetrics. Return 404 if requested ID not in system.
MR2800NBI objectmetrics. Return 404 if requested ID not in system.
MR2811whois: Do not update cache on download errors
MR2822Add batch work to fix_link_all_objects.
MR2832DataStream client: Handle DCS ResolutionError properly
MR2832DataStream client: Handle DCS ResolutionError properly
MR2836Fix service shutdown on nsq topics shutdown timeout
MR2845noc/noc#1139 Fix _write_int on ber.pyx for 64-bit value.
MR2851BBox checking
MR2876Fix initial_data["pool"] trace when managedobject change Pool.
MR2880Add cert param to ConsulHTTPClient (addition version 1.1.0).
MR2887Fix escalation closed while escalated proccessed.
MR2942noc/noc#1094 Add drop old maintenance collections.
MR2956noc/noc#1155 Catch LDAPCommunicationError and LDAPServerPoolExhaustedError LDAP server error.
MR2957Fix typo in xmac discovery.
MR2966Add ValueError to catch cards confdb error.
MR2973Fix Invalidate credentials cache when object_profile change.
MR2978noc/noc#1153 Fix config value if consul set it to empty string "".
MR3003Fix UserProfile import on wipe command.
MR3005#1102 Fix RouterOS tokenizer

Code Cleanup

MR2445docs: Source code documentation build
MR2480Start NOC 19.4 release cycle
MR2505move lib/convert.py to core/convert/dbm.py. fix profiles
MR2514move lib/dateutils.py to core/dateutils.py.
MR2515move lib/escape.py to core/escape.py
MR2516move lib/timepattern.py to core/timepattern.py
MR2537move lib/geo.py to core/geo
MR2544move lib/url to core/url
MR2585move lib/forms to core/forms
MR2589move lib/rpsl to core/rpsl
MR2601move lib/prettyjson to core/prettyjson
MR2611move lib/text to core/text. add tests
MR2615bump bootstrap to 3.3.7 cause of web.json already have one
MR2617move lib/validators to core/validators
MR2626remove lib/url.py
MR2654Add migration statements for release notes
MR2709Fix Clear Alarm, when option disabled in alarm class
MR2774Remove deprecated config section
MR2878Geocoders moved to noc.core.geocoder
MR2893py3 compatibility fixes
MR2894py3: Fix base64 usage
MR2895py3: zlib fixes
MR2896py3: Replace logger.warn() with logger.warning()
MR2897Profile re r cleanup part1
MR2898py3: hashlib fixes
MR2899py3: Fix print usage
MR2900Cleanup app file on re.
MR2901Profile re r cleanup part2
MR2902py3: BER fixes
MR2903py3: Fix CH charding
MR2905Remove NOC 19.4 deprecations
MR2907py3: Fix md5crypt
MR2909py3: Fix fm_escape/fm_unescape
MR2912py3: Fix cmp() usage
MR2913Fix Generic.get_metrics script
MR2916py3: unicode() fixes
MR2918Profile re r cleanup part3
MR2923dict() optimization
MR2925py3: GridVCS fixes
MR2926py3: SNMP Fixes
MR2928Fix typo on smart_text argument position.
MR2930py3: Fix ConfDB
MR2931py3: Fix SNMP TC processing
MR2932py3: Fix model unicode test
MR2933Cleanup regex errors on profile.
MR2936py3: Fix telnet
MR2939py3: unicode check fixes
MR2941py3: Fix DNS
MR2944py3: unicode check fixes.
MR2948py3: Fix cli
MR2949py3: Fix BasicAuth test
MR2950py3: Fix NSQ mpub
MR2954py3: Customized pyclips
MR2955py3: Fix main.desktop test
MR2963py3: Fix replace_re_group
MR2967Fix \x symbol on strip.
MR2977Fix 'Overlong 2 byte UTF-8 sequence detected when encoding string' errors on activator methods.
MR2982py3: Various fixes
MR2984py3: Translation fixes
MR2985py3: Login service fixes.
MR2986py3: config string params fix.
MR2987py3: Fix pickle field.
MR2990py3. Fix beef working.
MR2992Fix limit param check on extapplication.
MR2993py3 fix chwriter records type.
MR2994Py3. Fix build_echo_request.
MR2996Fix cleanup-pyc.
MR3002Fix Django version on docker requirements.

Profile Changes


MR2590Update Alstec.ALS profile
MR2784Alstec.24xx.get_interfaces. Add name port format.
MR2796Alstec.24xx. Add get_inventory script.
MR2884Fix prompt in Alstec.24xx profile for exclude banner


MR2491Add Axis.VAPIX confdb normalizer.
MR2500Fix Axis.VAPIX profile API parser when comment in output.
MR2520Axis.VAPIX. Fix normalize_resolution in confdb normalizer.


MR2525Fix BDCOM.xPON profile
MR2733Add BDCOM.IOS.get_inventory script
MR2820BDCOM.IOS.get_mac_address_table. Fix colected interface on mac table.
MR2820BDCOM.IOS.get_mac_address_table. Fix colected interface on mac table.


MR2788Add mpls l2vp to Cisco profile.
MR2790Fix Cisco vLAN subif ifindex matching
MR2794Add Cisco IOS SNMP metrics for PPPoE and CPU Load
MR2883Add suport for WS-C4900M switch to Cisco.IOS.get_inventory script
MR2961Update Cisco.IOS profile
MR2965Cisco.IOS. Fix confdb normalizer interface speed error.
MR2974Cisco.IOS.get_switchport. Fix vlan_id 0 on SNMP output.
MR2989Cisco.IOS.get_cdp_neighbors. Add execute_snmp method.
MR2995Cisco.IOS. confdb normalizer fix range notation for vlan.


MR2814Dlink.DxS_Smart add DXS1210
MR2861fix dlink_cisco_cli get_version


MR2456Fix typo in Dahua.DH confdb normalizer.
MR2524Add RVi vendor to Dahua.DH profile.
MR2557Dahua.DH. Add Web Auth middleware abd getting PTZ version to attributes.


MR2392Fix EdgeCore.ES.get_version script
MR2466Resolve "ConfDB: EdgeCore-ES ntp"
MR2522Edcore.ES.get_switchport. Fix 'interface_swport' regex if output not 'Native VLAN' field
MR2538confdb edgecore-es. fix ntp address
MR2691Fix EdgeCore.ES.get_lldp_neighbors script
MR2789EdgeCore.ES.get_lldp_neighbors. Fix port_id local converting bug.


MR2394Add line_wrapper and row_wrapper arguments to parse_table function
MR2471Resolve "ConfDB: Eltex-MES ntp"
MR2484Bring matchers to Eltex.MES profile
MR2511Fix method name on Eltex.MES confdb normalizer.
MR2569Fix Eltex.MES.get_inventory script
MR2597Fix Eltex.MES.get_capabilities script
MR2662Add support for Eltex.MES24xx profile
MR2701Fix Eltex.MES.get_lldp_neighbors script
MR2723Fix Eltex mes24xx profile
MR2781Add Eltex.MES24xx.get_lldp_neighbors script
MR2792Fix Eltex.MES.get_inventory script
MR2830Eltex.MES Fix confdb normalizer traceback.
MR2877Eltex.MES. Clean log message on console after login.
MR2882Fix Eltex.MES.get_interfaces script
MR2988Eltex.MES.get_lldp_neighbors. Move snmp to generic script.


MR2873Extreme.ISW. Add new SA profile.


MR2668Fix Extreme.XOS.get_portchannel script
MR2802Fix Extreme.XOS.get_interfaces script
MR2980Extreme.XOS. Fix detecting stack for X480-24x model.


MR2689Add interface errors delta metrics.
MR2715Generic. Add network stp topology changes delta metric.
MR2767Add "Network | STP" check to STP Topology metrics.
MR2783Add ability to get port name from another table
MR2865Generic.get_mpls_vpn. Fix rd param as list.
MR2919Add revision into Generic.get_inventory script
MR2968Generic.get_inventory. Fix KeyError if not 'HW version'.
MR2975Generic.get_capabilities. Add 'SNMP | OID | EnterpriseID' caps.
MR2976Generic.get_lldp_neighbors.Fix utf-8 decode.
MR2998Generic.get_beef improve worked.


MR2435Fix Hikvision.DSKV8 channel_name
MR2465Fix empty text on Hikvision.DSKV8.get_config.
MR2523Hikvision.DSKV8. Fix trace when returned bad xml.


MR2477Fix mailformed ADSL interface name in Huawei.MA5600T.get_interfaces.
MR2573Fix get_metrics MA5600T
MR2697Huawei.MA5600T.get_inventory. Add getting daighter card for MA5600.
MR2704Huawei.MA5600T add xDSL metrics type.
MR2706Huawei.MA5600T.get_inventory. Fix ma5626g chassis.


MR2482ConfDB: Allow over-dedent in indent tokenizer
MR2485Fix Huawei.VRP matchers for S5628F model.
MR2517Fix Huawei.VRP "Are you sure" pattern_more.
MR2635Fix STP detection in Huawei.VRP.get_capabilities script
MR2635Fix STP detection in Huawei.VRP.get_capabilities script
MR2777Huawei.VRP.get_mpls_vpn. Fix shifting string output trace.
MR2787Huawei.VRP. Add mpls l2vpn to profile.
MR2801Huawei.VRP.get_version. Fix CX600 detect version.
MR2809Add Huawei Entity extent mib.
MR2809Add Huawei Entity extent mib.
MR2818Huawei.VRP.get_inventory. Use display elabel command.
MR2821Huawei.VRP.get_inventory. Add interface binding to L2 MPLS.
MR2831Huawei.VRP. Fix vlan batch parser on confdb normalizer.
MR2870Huawei.VRP.get_lacp_neighbors. Fix trace on bad table.
MR2971noc/noc#1082 Huawei.VRP.confdb. Add parse NTP client settings.


MR2431Add pattern_operation_error to Iskratel.MSAN profile.
MR2739Iskratel.MSAN. Add xDSL metrics.


MR2470Resolve "ConfDB: Juniper-JUNOS ntp"
MR2659Fix Juniper.JUNOS.get_inventory script
MR2672Fix Juniper.JUNOS.get_inventory script
MR2681Add SIB connector type for Juniper EX8200 switches
MR2711Add more types in Juniper.JUNOS.get_inventory script
MR2771Fix Juniper.JUNOS.get_inventory script
MR2858Juniper.JunOS.get_interfaces. Add VPLS,VLL interface mapping.


MR2454Resolve "ConfDB: RouterOS ntp"
MR2645Update MikroTik.RouterOS.get_inventory.py: add get transceivers info.
MR2695Update MikroTik.RouterOS.get_license.py: fix error on CHR platform.


MR2410Update NAG.SNR.get_portchannel.py
MR2417Fix pager in NAG.SNR profile
MR2462Update NAG.SNR profile
MR2540Add nag.snr.get_inventory
MR2658Fix NAG.SNR.get_version script
MR2702Update NAG.SNR.get_inventory.py


MR2430Delete include from command in Qtech.QSW2800 profiles.
MR2444Update Qtech.QSW8200 profile
MR2468Resolve "ConfDB: Qtech-QSW2800 ntp"
MR2539confdb qtech_qsw2800. fix ntp address
MR2728Add Qtech.QSW8200.get_inventory script
MR2815Qtech.QSW2800.get_inventory. Fix trace when return result without 'HW version'.
MR2815Qtech.QSW2800.get_inventory. Fix trace when return result without 'HW version'.
MR2981Qtech.QSW2800.get_capabilities. Add has_stp_snmp method.


MR2476Add Rotek vendor to Raisecom.ROS profile.
MR2581Raisecom.ROS.get_interfaces. Fix getting iface_type.
MR2866Raisecom.ROS. Fix detect interface type profile method.
MR2945Raisecom.ROS. Fix working for Rotek switches.
MR2951noc/noc#1162 Raisecom.ROS. Fix get_lldp_neighbors port regex.


MR2616Update SKS.SKS profile
MR2683Fix prompt at SKS.SKS profile
MR2812SKS.SKS.get_interfaces. Fix stuck CLI command when discovery run.


MR2863fix Siklu get_inventory


MR2744Update Ubiquiti.AirOS profile


MR2743Fix Zyxel.MSAN profile
MR2752Fix Zyxel.MSAN.get_interfaces script


MR2401Add support ESCOM L to Iskrate.ESCOM profile.
MR2425Fix dlink_smart interfaces output
MR2531Update Extreme.Summit200 profile
MR2534Add new profile TP-Link EAP
MR2550Update Angtel.Topaz profile
MR2578Update Alstec.MSPU profile
MR2603DCN.DCWL.get_interfaces. Filter "00:00:00:00:00:00" MAC.
MR2619Update Zhone.MXK profile
MR2633fix cisco smb get switchport
MR2644Update Upvel.UP profile
MR2661Fix Upvel.UP.get_inventory script
MR2667Fix vlan parsing in BDCOM.xPON.get_interfaces script
MR2676Raisecom.RCIOS add get_inventory.
MR2694Eltex.WOP.get_metrics - Disable interface metrics on CLI.
MR2705Eltex.DSLAM. Add base xDSL metrics.
MR2708Fix Infinet get_lldp_neighbors and get_capabilities
MR2720Eltex.WOP.get_metrics. Fix Radio | Tx Power metric.
MR2731Fix Eltex.MA4000.get_inventory script
MR2732Add Vitesse.VSC.get_inventory script
MR2737Fix Vitesse.VSC.get_interfaces script
MR2738DLink.DAS. Add xDSL metrics and get_inventory script.
MR2740Add base support ECI.HiFOCuS profile.
MR2746Add Eltex.RG.get_inventory script
MR2759Rotek.RTBSv1 profile SNMP Prefer.
MR2776Update Zhone.Bitstorm profile
MR2803Upvel.UP. Add get_interface_names to profile for LLDP.
MR2803Upvel.UP. Add get_interface_names to profile for LLDP.
MR2804Alcatel.7302. Add get_inventory script.
MR2852ECI.HiFOCuS. Add get_inventory, Improvement SNMP work.
MR2853add Nateks NetXpert get_inventory
MR2854add H3C VRP get_inventory
MR2864Rotek.RTBS.get_interfaces. Fix traceback when unknown iface type.
MR2872Eltex.LTP. Add SNMP support.
MR2885Add Intracom docs.
MR2920Fix some get_version script to unify attribures
MR2927noc/noc#1159 Add support for MikroTik.SwOS profile
MR2970noc/noc#1183 DLink.DAS.get_inventory. Fix serial number regex.
MR2983ECI.HiFOCus. SNMP method prefer for working.
MR2999ECI.HiFOCus. Prefer use SNMP.
MR3000Eltex.LTE. Add operation error.