NOC 19.4


In accordance to our Release Policy we’re proudly present release 19.4.

19.4 release contains of 394 bugfixes, optimisations and improvements. Refer to the Release Notes for details.


Python3 Compatibility Statement

Python3 compatibility was one of primal focuses during 19.4 release cycle. We made lots of code reviews and cleanups to improve py3 compatibility. NOC 19.4 is able to perform test suite both on Python 2.7 and Python 3.6 and most of functions work on py3, though some glitches still remain. So we continue routine work on reaching full py3 compatibility.

We expect that:

  • NOC 20.1 will have full py3 compatibility and will allow to choose between py2 and py3 during deploy.
  • NOC 20.2 will by py3-only release with all py2-compatibility stuff removed.
  • NOC 20.3 will use py3 features like async/await and will switch from tornado to FastAPI.

Extended MAC discovery

Discovery got extended MAC (xmac) discovery method, showing good results in linking dumb terminal equipment like CPE’s, IP Phones, IP cams, which have no other reliable discovery methods.

See #1071 for details.

DataStream improvements

IPAM got VRF, Prefix and Address streams allowing to expose all IPAM data to external systems.

managedobject DataStream got additional tags attribute exposing ManagedObjects’s tags. Tags allow to pass unstructured metadata to external system.

managedobject DataStream also exposes comprehensive VPN information.

nbi improvements

NBI got 3 additional groups of API:

  • objectstatus for online Managed Object status requesting
  • config and configrevisions for accessing Managed Object’s config
  • path - the swiss-army knife for network topology paths calculation

time_delta in metrics

Discovery writes time elapsed from previous check to time_delta field, allowing to store metrics deltas instead of speed, allowing to perform tasks like traffic calculation and robust error detection.

Objects can be searched and centered on network map which can be helpful on large maps.

Compare Configs

Configs from different managed objects can be compared via UI

BI improvements

Amount of reboots, outages, STP topology changes are added to Managed Objects’ BI statistics allowing additional problem drill-down vectors.

ICQ sender

Additional icqsender service allows to push notifications via ICQ messenger.

Development Process Changes

py3 tests

Python3 tests are mandatory now and all test suite must pass python3 testings stages as well.

Mypy integration

NOC adopts Mypy. Though mypy tests are still advisory developers are urged to add type annotations to new and to the refactored code.

Code Formatting

NOC adopts Prettier <>_ for JS, JSON, CSS and YAML code formatting. CI pipeline checks code formatting of changed files. Any misformatting considered the error and CI pipeline fails at the lint stage. We recommend to add Prettier formatting to git’s pre-commit hook or to the IDE’s on-save hook.

Docker container is also available. Use::

docker run --rm \
    -w /src \
    -v $PWD:/src \ \
    /usr/local/bin/prettier --config=.prettierrc --write <file name>

to format file

New features

MR Title
!2234 network segment: search & select object
!2374 Took default user password and email from env on initial migrations
!2532 nbi: config and configrevisions API
!2613 metrics: Collect time_delta parameter
!2655 DataStream: ManagedObject tags
!2671 Config diff from two objects’ revisions
!2714 noc/noc#1126 Add n_reboots, availability, total_unavailability, n_outages,…
!2727 Path API
!2764 DataStream: Expose VPN information to managedobject datastream
!2772 Add ICQ sender notification service
!2862 Extended MAC discovery
!2881 datastream: IPAM integration
!2886 nbi: objectstatus API


MR Title
!1787 Add full address on map
!1990 Replace pylibssh2 with ssh2-python
!2408 docs: Fix profile anchors
!2415 ConfDB: Syntax documentation
!2492 Add discovery status classificator to report object stat.
!2504 Check for nbi MR label
!2506 cmtext height 100%
!2508 Filter enable_discovery False from Report Profile Check Summary.
!2533 login: Log authentication failures
!2548 Change to Edit in ModelApplication.js title
!2562 translate syslog
!2567 Set secure and httponly cookie
!2571 prepare for build
!2574 Prettier code formatting check
!2575 pynsq 0.8.3
!2576 Django 1.11.24
!2577 pyfilesystem2 2.4.10
!2591 NSQ http publisher
!2594 escalation: Configurable group tt promotion
!2598 Add
!2602 Add IPv4 address format check to ManagedObject etl loader.
!2609 Optimize uplink calculations for complex topologies
!2614 #955 sa.managedobject: Group editing of description
!2625 ClickHouse: Insert into JSONEachRow format
!2627 python-consul 1.1.0
!2636 Add ignore_empty_cli_response to script cli method.
!2638 Add loader to GeocoderCache.
!2656 sa.managedobject: Additional access checks in the config preview
!2669 Cleanup CH sharding code
!2673 JSONL format for BI ETL
!2693 Add ObjectProfile field to Alarm and Reboot BI models.
!2712 Add arguments to BI extract method.
!2717 Fix the name of the standard for Gigabit Ethernet.
!2718 pytest 4.6.6, Coverage 4.5.4, removed unused option ./noc test –test-report
!2719 typing
!2722 Extend logging on MIB import.
!2730 sa.managedobjectprofile, inv.interfaceprofile: Case-insensitive substring search
!2742 maintenance networksegment type changed from lookup to tree
!2754 MO profile, numberfield minValue set
!2762 ConfDB: Data Types
!2763 ConfDB: MPLS protocol
!2791 RepoPreview style fixed
!2799 Add ICQ sender service docs
!2813 modelfilter.List remove padding: 10
!2823 Add segment mac to discovery command.
!2824 Add forwarding_instance index to subinterface model.
!2825 Add config.version_format for noc version customization.
!2829 Extend logging when ConsulDCS lost session.
!2846 Fix icqsender doc
!2855 Add SNMP encode_oid function speedup.
!2871 Add ObjectProfile fieild to ManagedObject BI model.
!2889 fix duration for active alarms
!2892 CI: Run python3 tests automatically
!2906 Speedup ReportAvailability.
!2908 Add url storage path to beef command
!2921 noc/noc#1159 Add eof_mark param and extend logging on digestauth.
!2924 noc/noc#1157 Add filter_vendor, filter_platform, filter_version to ManagedObjectSelector application.
!2929 Add new get inventory2
!2934 Add description param to beef collect command.
!2938 Add ad-hoc beef options to script
!2940 PyFilesystem2 2.4.11
!2943 noc/noc#1104 Add static MAC to mac discovery.
!2947 Update HP.ProCurve profile
!2958 noc/noc#1158 Disconnect connections on asset discovery.
!2959 py3: Set py3 tests as mandatory
!2962 Replace interface profile on nri_service discovery when change settings.
!2964 noc/noc#1088 Add ifindex and mac to confdb interface meta.
!2969 Add to beef command allow use local specs.
!2972 Add ERR_SNMP_BAD_COMMUNITY error when credentials without SNMP.
!2979 noc/noc#1098 Remove read() from read json file in inject-event command.
!2997 Speedup DiscoveryTopology Problem report.
!3001 Use sys.platform unstead os.uname().


MR Title
!2281 Fix HP.ProCurve.get_lldp_neighbors script
!2446 Add DISABLE_SERVER_SIDE_CURSOR params to connect django db.
!2463 form objectvalidationpolicy fixed
!2464 Fix KB index page trace when empty user history.
!2481 Remove newline from version info
!2486 Fix interface-profile command trace when connect.
!2490 Fix ddash when no metric on interface profile
!2501 Fix matcher parsing
!2503 Ensure User.last_login is nullable
!2535 right width increase
!2536 Fix on_init attribute on administrative_domain model.
!2545 Use ReportModelFilter for ReportDiscoveryLinks.
!2547 Change user_permissions field to permissions (fix create user).
!2549 Add ch_escape to managedObject BI extractor.
!2558 noc/noc#1095 Fix mongo connect on commands job, run, wipe.
!2559 Fix DiscoveryResult reportdatasource high memory consumption.
!2560 Register unknown part_no use first vendor_code
!2568 Add object_profile field to managed_object iter_datastream.
!2570 translate Add Insert for listformfield and gridfield fixed
!2572 Add use_mongo to mrt services.
!2587 #1105 Fix apply fix_object_uplinks and fix_alarm_managedobjectprofile
!2595 Fix ReportObjectDetail append row.
!2600 #1100 Fix FirmwarePolicy.get_recommended_version()
!2612 collections: Ignore unknown fields
!2618 Update commands/
!2621 Fix mongo connect on whois command.
!2622 Fix clear unknown model when create.
!2629 Add UUID to JSON Unique field on Inventory models.
!2634 #1112 fix csv-import/export command
!2637 Fix interface_flap param on ReportMetric.
!2660 Fix broke –clean argument on events command
!2666 Fix ‘code’ object is not callable on metric shard function.
!2696 Fix syslogcollector.register_message typo.
!2699 requirements tablesorter & table2csv added
!2700 #1122 Split too large published messages into parts
!2710 Fix CSV Import
!2713 noc/noc#1122 Workaround for stuck metrics queue when message very large.
!2716 Fix _iter_metrics_raw_chunks chunk limit config.
!2747 Handle NSQ publisher failures
!2765 Fix escalation Stop Processing
!2766 Set availability ManagedObject BI extractor to 100%.
!2773 Chrome ignores autocomplete=‘off’ - fixed
!2798 Catch SyntaxError trace when managedObject card open.
!2800 NBI objectmetrics. Return 404 if requested ID not in system.
!2800 NBI objectmetrics. Return 404 if requested ID not in system.
!2811 whois: Do not update cache on download errors
!2822 Add batch work to fix_link_all_objects.
!2832 DataStream client: Handle DCS ResolutionError properly
!2832 DataStream client: Handle DCS ResolutionError properly
!2836 Fix service shutdown on nsq topics shutdown timeout
!2845 noc/noc#1139 Fix _write_int on ber.pyx for 64-bit value.
!2851 BBox checking
!2876 Fix initial_data[“pool”] trace when managedobject change Pool.
!2880 Add cert param to ConsulHTTPClient (addition version 1.1.0).
!2887 Fix escalation closed while escalated proccessed.
!2942 noc/noc#1094 Add drop old maintenance collections.
!2956 noc/noc#1155 Catch LDAPCommunicationError and LDAPServerPoolExhaustedError LDAP server error.
!2957 Fix typo in xmac discovery.
!2966 Add ValueError to catch cards confdb error.
!2973 Fix Invalidate credentials cache when object_profile change.
!2978 noc/noc#1153 Fix config value if consul set it to empty string “”.
!3003 Fix UserProfile import on wipe command.
!3005 #1102 Fix RouterOS tokenizer

Code Cleanup

MR Title
!2325 fix_mib_make-cmib
!2445 docs: Source code documentation build
!2480 Start NOC 19.4 release cycle
!2505 move lib/ to core/convert/ fix profiles
!2514 move lib/ to core/
!2515 move lib/ to core/
!2516 move lib/ to core/
!2537 move lib/ to core/geo
!2544 move lib/url to core/url
!2585 move lib/forms to core/forms
!2589 move lib/rpsl to core/rpsl
!2601 move lib/prettyjson to core/prettyjson
!2611 move lib/text to core/text. add tests
!2615 bump bootstrap to 3.3.7 cause of web.json already have one
!2617 move lib/validators to core/validators
!2626 remove lib/
!2654 Add migration statements for release notes
!2709 Fix Clear Alarm, when option disabled in alarm class
!2774 Remove deprecated config section
!2878 Geocoders moved to noc.core.geocoder
!2893 py3 compatibility fixes
!2894 py3: Fix base64 usage
!2895 py3: zlib fixes
!2896 py3: Replace logger.warn() with logger.warning()
!2897 Profile re r cleanup part1
!2898 py3: hashlib fixes
!2899 py3: Fix print usage
!2900 Cleanup app file on re.
!2901 Profile re r cleanup part2
!2902 py3: BER fixes
!2903 py3: Fix CH charding
!2905 Remove NOC 19.4 deprecations
!2907 py3: Fix md5crypt
!2909 py3: Fix fm_escape/fm_unescape
!2912 py3: Fix cmp() usage
!2913 Fix Generic.get_metrics script
!2916 py3: unicode() fixes
!2918 Profile re r cleanup part3
!2923 dict() optimization
!2925 py3: GridVCS fixes
!2926 py3: SNMP Fixes
!2928 Fix typo on smart_text argument position.
!2930 py3: Fix ConfDB
!2931 py3: Fix SNMP TC processing
!2932 py3: Fix model unicode test
!2933 Cleanup regex errors on profile.
!2936 py3: Fix telnet
!2939 py3: unicode check fixes
!2941 py3: Fix DNS
!2944 py3: unicode check fixes.
!2948 py3: Fix cli
!2949 py3: Fix BasicAuth test
!2950 py3: Fix NSQ mpub
!2954 py3: Customized pyclips
!2955 py3: Fix main.desktop test
!2963 py3: Fix replace_re_group
!2967 Fix \x symbol on strip.
!2977 Fix ‘Overlong 2 byte UTF-8 sequence detected when encoding string’ errors on activator methods.
!2982 py3: Various fixes
!2984 py3: Translation fixes
!2985 py3: Login service fixes.
!2986 py3: config string params fix.
!2987 py3: Fix pickle field.
!2990 py3. Fix beef working.
!2992 Fix limit param check on extapplication.
!2993 py3 fix chwriter records type.
!2994 Py3. Fix build_echo_request.
!2996 Fix cleanup-pyc.
!3002 Fix Django version on docker requirements.

Profile Changes


MR Title
!2590 Update Alstec.ALS profile
!2784 Alstec.24xx.get_interfaces. Add name port format.
!2796 Alstec.24xx. Add get_inventory script.
!2884 Fix prompt in Alstec.24xx profile for exclude banner


MR Title
!2491 Add Axis.VAPIX confdb normalizer.
!2500 Fix Axis.VAPIX profile API parser when comment in output.
!2520 Axis.VAPIX. Fix normalize_resolution in confdb normalizer.


MR Title
!2525 Fix BDCOM.xPON profile
!2733 Add BDCOM.IOS.get_inventory script
!2820 BDCOM.IOS.get_mac_address_table. Fix colected interface on mac table.


MR Title
!2788 Add mpls l2vp to Cisco profile.
!2790 Fix Cisco vLAN subif ifindex matching
!2794 Add Cisco IOS SNMP metrics for PPPoE and CPU Load
!2883 Add suport for WS-C4900M switch to Cisco.IOS.get_inventory script
!2961 Update Cisco.IOS profile
!2965 Cisco.IOS. Fix confdb normalizer interface speed error.
!2974 Cisco.IOS.get_switchport. Fix vlan_id 0 on SNMP output.
!2989 Cisco.IOS.get_cdp_neighbors. Add execute_snmp method.
!2995 Cisco.IOS. confdb normalizer fix range notation for vlan.
MR Title
!2814 Dlink.DxS_Smart add DXS1210
!2861 fix dlink_cisco_cli get_version


MR Title
!2456 Fix typo in Dahua.DH confdb normalizer.
!2524 Add RVi vendor to Dahua.DH profile.
!2557 Dahua.DH. Add Web Auth middleware abd getting PTZ version to attributes.


MR Title
!2392 Fix EdgeCore.ES.get_version script
!2466 Resolve “ConfDB: EdgeCore-ES ntp”
!2522 Edcore.ES.get_switchport. Fix ‘interface_swport’ regex if output not ‘Native VLAN’ field
!2538 confdb edgecore-es. fix ntp address
!2691 Fix EdgeCore.ES.get_lldp_neighbors script
!2789 EdgeCore.ES.get_lldp_neighbors. Fix port_id local converting bug.
!2910 add_snmp_Edgecore.ES_profile
!2953 fix_get_portchannel_Edge-Core_profile


MR Title
!2394 Add line_wrapper and row_wrapper arguments to parse_table function
!2471 Resolve “ConfDB: Eltex-MES ntp”
!2484 Bring matchers to Eltex.MES profile
!2511 Fix method name on Eltex.MES confdb normalizer.
!2569 Fix Eltex.MES.get_inventory script
!2597 Fix Eltex.MES.get_capabilities script
!2662 Add support for Eltex.MES24xx profile
!2701 Fix Eltex.MES.get_lldp_neighbors script
!2723 Fix Eltex mes24xx profile
!2750 fix_eltex_mes24xx_enable_mode
!2781 Add Eltex.MES24xx.get_lldp_neighbors script
!2792 Fix Eltex.MES.get_inventory script
!2830 Eltex.MES Fix confdb normalizer traceback.
!2875 fix_Environment-Temperature_Eltex.MES
!2877 Eltex.MES. Clean log message on console after login.
!2882 Fix Eltex.MES.get_interfaces script
!2988 Eltex.MES.get_lldp_neighbors. Move snmp to generic script.


MR Title
!2873 Extreme.ISW. Add new SA profile.


MR Title
!2668 Fix Extreme.XOS.get_portchannel script
!2802 Fix Extreme.XOS.get_interfaces script
!2980 Extreme.XOS. Fix detecting stack for X480-24x model.


MR Title
!2689 Add interface errors delta metrics.
!2715 Generic. Add network stp topology changes delta metric.
!2767 Add “Network
!2783 Add ability to get port name from another table
!2865 Generic.get_mpls_vpn. Fix rd param as list.
!2904 new_get_inventory_Generic_profile
!2919 Add revision into Generic.get_inventory script
!2937 fix_get_portchannel_Generic_profile
!2968 Generic.get_inventory. Fix KeyError if not ‘HW version’.
!2975 Generic.get_capabilities. Add ‘SNMP
!2976 Generic.get_lldp_neighbors.Fix utf-8 decode.
!2998 Generic.get_beef improve worked.


MR Title
!2435 Fix Hikvision.DSKV8 channel_name
!2465 Fix empty text on Hikvision.DSKV8.get_config.
!2523 Hikvision.DSKV8. Fix trace when returned bad xml.


MR Title
!2477 Fix mailformed ADSL interface name in Huawei.MA5600T.get_interfaces.
!2573 Fix get_metrics MA5600T
!2697 Huawei.MA5600T.get_inventory. Add getting daighter card for MA5600.
!2704 Huawei.MA5600T add xDSL metrics type.
!2706 Huawei.MA5600T.get_inventory. Fix ma5626g chassis.


MR Title
!2482 ConfDB: Allow over-dedent in indent tokenizer
!2485 Fix Huawei.VRP matchers for S5628F model.
!2517 Fix Huawei.VRP “Are you sure” pattern_more.
!2635 Fix STP detection in Huawei.VRP.get_capabilities script
!2635 Fix STP detection in Huawei.VRP.get_capabilities script
!2777 Huawei.VRP.get_mpls_vpn. Fix shifting string output trace.
!2787 Huawei.VRP. Add mpls l2vpn to profile.
!2801 Huawei.VRP.get_version. Fix CX600 detect version.
!2809 Add Huawei Entity extent mib.
!2809 Add Huawei Entity extent mib.
!2818 Huawei.VRP.get_inventory. Use display elabel command.
!2821 Huawei.VRP.get_inventory. Add interface binding to L2 MPLS.
!2831 Huawei.VRP. Fix vlan batch parser on confdb normalizer.
!2870 Huawei.VRP.get_lacp_neighbors. Fix trace on bad table.
!2971 noc/noc#1082 Huawei.VRP.confdb. Add parse NTP client settings.


MR Title
!2431 Add pattern_operation_error to Iskratel.MSAN profile.
!2739 Iskratel.MSAN. Add xDSL metrics.


MR Title
!2470 Resolve “ConfDB: Juniper-JUNOS ntp”
!2659 Fix Juniper.JUNOS.get_inventory script
!2672 Fix Juniper.JUNOS.get_inventory script
!2681 Add SIB connector type for Juniper EX8200 switches
!2711 Add more types in Juniper.JUNOS.get_inventory script
!2771 Fix Juniper.JUNOS.get_inventory script
!2858 Juniper.JunOS.get_interfaces. Add VPLS,VLL interface mapping.


MR Title
!2454 Resolve “ConfDB: RouterOS ntp”
!2645 Update add get transceivers info.
!2695 Update fix error on CHR platform.


MR Title
!2410 Update
!2417 Fix pager in NAG.SNR profile
!2462 Update NAG.SNR profile
!2540 Add nag.snr.get_inventory
!2658 Fix NAG.SNR.get_version script
!2702 Update


MR Title
!2430 Delete include from command in Qtech.QSW2800 profiles.
!2444 Update Qtech.QSW8200 profile
!2468 Resolve “ConfDB: Qtech-QSW2800 ntp”
!2539 confdb qtech_qsw2800. fix ntp address
!2728 Add Qtech.QSW8200.get_inventory script
!2815 Qtech.QSW2800.get_inventory. Fix trace when return result without ‘HW version’.
!2815 Qtech.QSW2800.get_inventory. Fix trace when return result without ‘HW version’.
!2981 Qtech.QSW2800.get_capabilities. Add has_stp_snmp method.


MR Title
!2476 Add Rotek vendor to Raisecom.ROS profile.
!2581 Raisecom.ROS.get_interfaces. Fix getting iface_type.
!2866 Raisecom.ROS. Fix detect interface type profile method.
!2945 Raisecom.ROS. Fix working for Rotek switches.
!2951 noc/noc#1162 Raisecom.ROS. Fix get_lldp_neighbors port regex.


MR Title
!2616 Update SKS.SKS profile
!2683 Fix prompt at SKS.SKS profile
!2812 SKS.SKS.get_interfaces. Fix stuck CLI command when discovery run.


MR Title
!2863 fix Siklu get_inventory


MR Title
!2744 Update Ubiquiti.AirOS profile


MR Title
!2743 Fix Zyxel.MSAN profile
!2752 Fix Zyxel.MSAN.get_interfaces script


MR Title
!2312 Intracom.Ultralink
!2401 Add support ESCOM L to Iskrate.ESCOM profile.
!2425 Fix dlink_smart interfaces output
!2531 Update Extreme.Summit200 profile
!2534 Add new profile TP-Link EAP
!2550 Update Angtel.Topaz profile
!2578 Update Alstec.MSPU profile
!2603 DCN.DCWL.get_interfaces. Filter “00:00:00:00:00:00” MAC.
!2619 Update Zhone.MXK profile
!2633 fix cisco smb get switchport
!2644 Update Upvel.UP profile
!2661 Fix Upvel.UP.get_inventory script
!2667 Fix vlan parsing in BDCOM.xPON.get_interfaces script
!2676 Raisecom.RCIOS add get_inventory.
!2694 Eltex.WOP.get_metrics - Disable interface metrics on CLI.
!2705 Eltex.DSLAM. Add base xDSL metrics.
!2708 Fix Infinet get_lldp_neighbors and get_capabilities
!2720 Eltex.WOP.get_metrics. Fix Radio
!2731 Fix Eltex.MA4000.get_inventory script
!2732 Add Vitesse.VSC.get_inventory script
!2737 Fix Vitesse.VSC.get_interfaces script
!2738 DLink.DAS. Add xDSL metrics and get_inventory script.
!2740 Add base support ECI.HiFOCuS profile.
!2746 Add Eltex.RG.get_inventory script
!2759 Rotek.RTBSv1 profile SNMP Prefer.
!2776 Update Zhone.Bitstorm profile
!2803 Upvel.UP. Add get_interface_names to profile for LLDP.
!2803 Upvel.UP. Add get_interface_names to profile for LLDP.
!2804 Alcatel.7302. Add get_inventory script.
!2852 ECI.HiFOCuS. Add get_inventory, Improvement SNMP work.
!2853 add Nateks NetXpert get_inventory
!2854 add H3C VRP get_inventory
!2864 Rotek.RTBS.get_interfaces. Fix traceback when unknown iface type.
!2872 Eltex.LTP. Add SNMP support.
!2885 Add Intracom docs.
!2911 fix_get_inventory1
!2920 Fix some get_version script to unify attribures
!2927 noc/noc#1159 Add support for MikroTik.SwOS profile
!2970 noc/noc#1183 DLink.DAS.get_inventory. Fix serial number regex.
!2983 ECI.HiFOCus. SNMP method prefer for working.
!2999 ECI.HiFOCus. Prefer use SNMP.
!3000 Eltex.LTE. Add operation error.