26.08.2020
In accordance to our Release Policy we’re proudly present release 20.3.
20.3 release contains of 271 bugfixes, optimisations and improvements. Refer to the Release Notes for details.
Highlights
Biosegmentation
NOC 20.3 introduces new ad-hoc auto-segmentation algorithm we’d called Biosegmentation for its similarity with famous “Osmosys” game. Each Managed Object starts with its very own “floating” segment, or “bubble”. Bubbles collide each other in series of neighboring events, like new links found. Each collision leads to a “trial”, when rivaling segments have to prove their power and appeal to the policies and to the rules.
The winner of contest may consume all looser’s objects and gains ever more power. During to collision with already established network segment hierarchy, called “persistent” segment, bubble can ever join to one of existing segment, or to became persistent segment by itself.
Biosegmentation can effectively reveal network segment hierarchy by analyzing existing or newly found links.
ConfDB Improvements
ConfDB got new syntax support for:
- LAG
- AAA
- DNS
Normalizers have been covered by common testing framework, increasing overall code quality and developers productivity.
Stencil Overlays
Network map got capability to denote object’s shapes with additional small glyphs, basing on object, profiles or project settings. i.e. all objects related to particular project or to particular role in the network may have additional indication.
New features
MR | Title |
---|---|
!3478 | Sentry: Change client |
!3686 | add api for check is logged |
!3795 | sa.managedobject: Manual interface creation/deletion |
!3814 | noc/noc#1318 Add LAG to ConfDB syntax. |
!3841 | Network Map: Stencil overlays |
!3893 | Font and Glyph collections |
!3904 | ManagedObject: Project |
!3960 | Update Maintenance functional |
!3995 | Add New profile STerra Gate |
!4002 | Add new Profile Kontinent |
!4012 | Add group Object metrics in Card |
!4022 | Add metrics for Gate. |
!4118 | Biosegmentation |
!4154 | tag tree widget |
!4169 | Add Eltex MES24xx mib |
!4176 | Update gates metrics name |
!4179 | Fix graph_interface_load_lag.j2 view sum iface load |
Improvements
MR | Title |
---|---|
!3380 | Add some help hints |
!3506 | #1331 Unified requirements.txt |
!3815 | Add aggregated interface and LACP protocol to Interface and Caps Discovery. |
!3816 | noc/noc#1319 Add cli_retries_unprivileged_mode params. Fix retry unprivileged mode. |
!3842 | Add SysUptime metric type. |
!3843 | Set enable_timedelta to true on Interface PM scope. |
!3843 | Set enable_timedelta to true on Interface PM scope. |
!3849 | Add BI dashboards to collections. |
!3873 | noc/noc#1325 Add downlinks of uplink to rca_neighbors. |
!3879 | Replace get_fqdn script to Generic. |
!3892 | noc/noc#1327 Add DefaultInterfaceUntaggedVlanApplicator to ConfDB applicators. |
!3895 | Replace get_fqdn script to Generic #2 |
!3897 | datastream: asyncio waiter |
!3898 | Fix upload docs |
!3900 | atomicl 1.1.0 |
!3907 | add_new_objectmodels_Zyxel_MSAN |
!3916 | Add some cisco models |
!3918 | Merge branch ‘fix-docker-image-proj’ into ‘master’ |
!3921 | features.use_uvloop config option |
!3922 | add_objectmodels_cisco_c3560e |
!3933 | add_objectmodels_dlink_DAS-3224DC_1 |
!3934 | add_objectmodels_zhone_paradyne |
!3940 | add_objectmodels_Extreme_X480-24x |
!3941 | Add RCA type field to Alarms BI model. |
!3945 | Add batch to fix_object_uplinks. |
!3956 | add_objectmodels_ciscoASR1004_card |
!3964 | Disable action button in grid, when we have not permission |
!3965 | Allow empty platform for etl portmapper. |
!3968 | add_profilecheckrules_Eltex_TAU |
!3969 | add_profilecheckrules_SKS |
!3972 | inv.objectmodels: Cisco |
!3985 | How to share collections |
!3988 | Add howto to menu |
!3996 | inv.objectmodels: Eltex |
!4003 | inv.objectmodels: Extreme |
!4004 | Update PWR-550W-AC.json |
!4005 | inv.connectiontypes: Extreme |
!4006 | inv.objectmodels: Extreme |
!4007 | inv.objectmodels: Extreme |
!4017 | Update DGS-3420-28TC.json |
!4018 | inv.objectmodels: Juniper |
!4029 | Update JPSU-650W-AC-AFO.json |
!4030 | inv.connectiontypes: Juniper |
!4031 | inv.objectmodels: Juniper |
!4032 | inv.objectmodels: Juniper |
!4033 | ADd dependencies scan |
!4034 | inv.objectmodels: APC |
!4036 | inv.objectmodels: Cisco |
!4037 | inv.objectmodels: Cisco |
!4038 | inv.objectmodels: Cisco |
!4039 | inv.objectmodels: Cisco |
!4041 | inv.objectmodels: Cisco |
!4042 | inv.objectmodels: Cisco |
!4043 | inv.objectmodels: MikroTik |
!4044 | inv.objectmodels: Nateks |
!4045 | inv.objectmodels: Nateks |
!4046 | inv.objectmodels: Ubiquiti |
!4047 | inv.objectmodels: Ubiquiti |
!4051 | inv.objectmodels: Cisco |
!4053 | inv.objectmodels: Cisco |
!4055 | Remove useless options and add changes rule. |
!4057 | Add more platforms |
!4059 | Add segment_lookup for segment column on ReportObjectDetail. |
!4063 | enable dependency test only when file is changed |
!4064 | inv.objectmodels: Cisco |
!4065 | inv.objectmodels: Cisco |
!4067 | noc/noc#1346 Filter metric if it value eq SNMP_OVERLOAD_VALUE. |
!4071 | inv.connectionrules: Cisco |
!4072 | inv.connectiontypes: Cisco |
!4073 | inv.objectmodels: Cisco |
!4074 | inv.objectmodels: Cisco |
!4075 | inv.connectiontypes: Cisco |
!4076 | inv.connectiontypes: Cisco |
!4077 | inv.connectiontypes: Cisco |
!4083 | Update list of datastreams in fix_rebuild_datastream |
!4085 | etl: BaseExtractor refactoring |
!4086 | sa.profilecheckrules: DLink |
!4087 | sa.profilecheckrules: Huawei |
!4088 | sa.profilecheckrules: Huawei |
!4088 | sa.profilecheckrules: Huawei |
!4105 | inv.models.platform: Fix to_json sorting |
!4113 | sa.profilecheckrules: Eltex |
!4115 | sa.profilecheckrules: DCN |
!4116 | sa.profilecheckrules: DCN |
!4117 | sa.profilecheckrules: DCN |
!4125 | pymongo 3.11.0 |
!4137 | sa.profilecheckrules tests |
!4140 | noc/noc#1328 Add vlan_ids to unit ConfDB syntax. |
!4146 | fix_reportmaxmetrics |
!4148 | noc/noc#1342 Add VPN Metric Scope. |
!4149 | etl: Configurable compression |
!4150 | Fix get_version.py for DCWS-6028(R2) |
!4151 | noc/noc#1218 CLI: Distinct timeout error codes |
!4162 | #1362 ConfDB: Group expansion in curly tokenizer |
!4163 | noc/noc#1360 Add aaa to system confdb syntax. |
!4164 | noc/noc#1361 Add dns to protocols confdb syntax. |
!4165 | Add vlan_ids to interface discovery ConfDB source. |
!4166 | Fix ReportObjectDetail trace when empty segment name. |
!4167 | SNMP: Better oid wrap detection in GETNEXT |
!4170 | #1361 confdb: Juniper.JUNOS dns normalizer |
!4172 | confdb: Profile normalizer tests |
!4175 | ConfDB: Allow normalizer inheritance |
!4182 | Update uplinks if MergeDownlink status changed. |
!4183 | #1364 Biosegmentation: Configurable segment names |
!4184 | Remove autogenerated words |
!4185 | Move Tornado-specific parts into TornadoService |
!4190 | pynsq 0.9.0 |
!4192 | Allow to use model id in DocumentReferenceField |
!4208 | ConfDB: Expose profile to normalizer tests |
!4210 | Optimized run_sync |
!4212 | Fix UI style for ConfDB query name. |
!4212 | Fix UI style for ConfDB query name. |
!4215 | noc/noc#1365 Close CLI stream if ‘Connection reset’ receive. |
!4219 | Set black format to py37 |
!4220 | Move pylint configuration to pyproject.toml file |
!4224 | gitlab ci: Do not try to lint deleted files |
!4226 | Parallel build of docs. Disable API docs |
!4227 | Minimize line-length to 100 like in black |
Bugfixes
MR | Title |
---|---|
!3784 | Fix docker image build |
!3791 | Fix encoding on ReportLinkDetail. |
!3792 | Fix load cmibs from custom. |
!3797 | Fix decoding on interfacepath card. |
!3800 | Fix tags lookup search. |
!3801 | Remove double fm_pool from managedobject etl loader. |
!3831 | noc/noc#1322 Fix duplicate session when self.cli use on Profile.shutdown_session. |
!3860 | Fix fix_link_all_objects. |
!3862 | Add cpestatus field to ReportDiscoveryResult reportdatasource. |
!3867 | Fix ‘CronTab’ object is not an iterator trace. |
!3872 | Fix trap decode (‘use a string pattern on a bytes-like object’). |
!3875 | Fix ‘Juniper |
!3878 | Add filter id None values on datastream command. |
!3880 | Fix encoding on classifier fm_unescape function. |
!3890 | Fix typo on CollapseTagged Applicator. |
!3910 | Fix monmap card ‘not supported between instances of ‘int’ and ‘NoneType”. |
!3911 | Fix sorted key in ConfDB ‘interfaces X meta link’. |
!3942 | Fix sorted key in BI Dashboard access. |
!3943 | Add suppress_trace flag to IOLoopContext. |
!3953 | Disable check access_preference SNMP for get_version script. |
!3977 | Fix grafana templates id |
!4048 | noc/noc#1343 Use ListField for inheritance on PlainReferenceListField. Fixed extend change on field. |
!4056 | noc/noc#1345 Fix ‘RuntimeError: no running event loop’ on Datastream notify callback. |
!4080 | combobox fix |
!4090 | Fix getting object metrics for table without path column. |
!4096 | Set uuid when create BI dashboard. |
!4098 | Fix typo in pm.utils. |
!4101 | Revert PlainReferenceListField. |
!4124 | #1333 Change AS.asn type to bigint to extend valid 32-bit ASN range |
!4127 | noc/noc#1343 Use BaseList from mongoengine.base.datastructures for PlainReferenceListField value. |
!4128 | noc/noc#1344 Use smart_text on fix tags. |
!4129 | noc/noc#1337 Use smart_bytes on threshold picker load. |
!4139 | Generic.get_interfaces. Fix mtu None value trace. |
!4157 | noc/noc#1324 Catch exception when shutdown_session when remove session. |
!4158 | Fix call handler on cli object_parser. |
!4171 | docs: Fix actual arguments in tools |
!4173 | noc/noc#1350 Fix datastream rebuild None id trace. |
!4197 | Add ifdescr to ReporDiscoveryResult. |
!4207 | Fix table name for routingneighbors BI models. |
!4216 | field biosseg_floating_parent_segment fixed |
Code Cleanup
MR | Title |
---|---|
!3462 | cython language_level |
!3630 | Fix Fav Filter |
!3779 | Update VERSION |
!3835 | Remove pyclips |
!3845 | Cleanup cm.ObjecFact on managedobject on_delete_check. |
!3851 | Remove main.probehandler.LookupField |
!3896 | Remove OrderMap |
!3919 | Remove “import json” from noc.lib.app.site |
!3971 | Update iface handler |
!4126 | Remove remaining future imports |
!4134 | Replace collections.Iterable with typing.Iterable |
!4177 | Replace deprecated imp.new_module() function |
Profile Changes
Alcatel.7302
MR | Title |
---|---|
!3828 | Alcatel.7302.get_version. Fix platform detect. |
!4211 | Alcatel.7302. Fix for 7342 PON platform. |
Alsitec.24xx
MR | Title |
---|---|
!3821 | Alstec.24xx.get_fqdn. Cleanup print. |
Cisco.IOS
MR | Title |
---|---|
!3802 | Cisco.IOS. Add Dial-peer interface to convert_interface_name. |
!3846 | Add voice interface to Cisco.IOS.convert_interface_name. |
!3908 | Use MIB to get CDP status, instead of raw oid |
!3917 | Fix Cisco.IOS.get_inventory script |
!4066 | Fix Cisco.IOS.get_inventory script |
!4130 | noc/noc#1339 Cisco.IOS.normalizer. Fix digit multi on storm-control level command. |
!4147 | noc/noc#1339 Cisco.IOS.normalizer. Fix typo |
DLink.DxS
MR | Title |
---|---|
!4089 | Add DLink.DxS.get_metrics script |
Eltex.MES
MR | Title |
---|---|
!4204 | ConfDB Eltex.MES |
Eltex.MES24xx
MR | Title |
---|---|
!3950 | Eltex.MES24xx. Add support SNMP to profile. |
!4106 | Fix Eltex.MES24xx.get_lldp_neighbors script |
!4155 | Add metrics CPU/Memory for MES24xx |
!4181 | ConfDB. MES24xx. normalizer + tests |
Extreme.XOS
MR | Title |
---|---|
!4112 | Fix Extreme.XOS.get_inventory script |
Generic
MR | Title |
---|---|
!3781 | Generic.get_interfaces. Fix getting switchport vlans. |
!3819 | Generic.get_interfaces. Fix get_snmp_timeout method. |
!3834 | Generic.get_interface_properties. Add interface_filter method for ignore interface by name. |
!3855 | Fix invalid vlan number in Generic.get_mac_address_table script |
!3935 | Fix Generic.get_interfaces script |
!3944 | Generic.get_mac_address_table. Add get_iface_mapping method. |
!4109 | noc/noc#1349 Generic.get_interfaces. Fix trace when empty mtu |
!4111 | Fix Generic.get_mpls_vpn script |
!4202 | Generic.get_spanning_tree. Convert designated_port_id to hex if getting int. |
Hikvision.DSKV8
MR | Title |
---|---|
!3870 | Hikvision.DSKV8.get_config. Fix dict iteritems. |
Huawei.MA5600T
MR | Title |
---|---|
!2363 | Add Operation error to Huawei.MA5600T when message. |
!3796 | Huawei.MA5600T.get_capabilities set keep_cli_session to false. |
!3827 | Huawei.MA5600T.get_inventory. Fix detect MA5603. |
!3854 | Huawei.MA5600T.get_cpe_status. Fix decode serial. |
!3920 | Huawei.MA5600T. Add pattern_more prompt. |
!3991 | Fix CLI Huawei.MA5600T.get_lldp_neighbors |
!4061 | Huawei.MA5600T. Add Load OID for GPON interface metrics. |
!4103 | Huawei.MA5600T. Fix typo on gponports metric name. |
!4110 | Fix Huawei.MA5600T profile |
Huawei.VRP
MR | Title |
---|---|
!3780 | Fix Huawei vrp 40g interface support |
!3783 | Huawei.VRP.get_interfaces. Remove print. |
!3810 | Huawei.VRP. ConfDB normalizer for ipv6 link-local address. |
!3822 | Huawei.VRP. Add LAG interface to ConfDB normalizer. |
!3886 | Huawei.VRP. Add Gi to convert_interface_name. |
!3951 | Huawei.VRP. Fix several snmp only working problems. |
!4013 | noc/noc#1163 Huawei.VRP. Add rogue_chars to profile. |
!4142 | noc/noc#1163 Huawei.VRP.get_metrics. Use getnext for CX200 DuplexStatus. |
!4201 | Huawei.VRP. ConfDB normalizer. Add parse vlan_ids. |
Juniper.JUNOS
MR | Title |
---|---|
!3703 | fix for JUNOS: get_version.py - get serial number from CLI |
!3772 | Fix invalid output in JUNOS 16.1R6.7 |
!3853 | Fix Juniper.JUNOS.get_fqdn script |
!3936 | Fix script for only snmp connection |
!3937 | Fix script for only snmp connection2 |
!3939 | Add use of filter_interface() function |
!4024 | Fix Juniper.JUNOS.get_interfaces script |
MikroTik.RouterOS
MR | Title |
---|---|
!3902 | Add SNMP support for MikroTik.RouterOS.get_fqdn script |
NAG.SNR
MR | Title |
---|---|
!3857 | Fix NAG.SNR.get_version script |
Qtech.QSW
MR | Title |
---|---|
!3905 | new_objectmodels_qtech_part4 |
Raisecom.ROS
MR | Title |
---|---|
!3832 | Raisecom.ROS.get_version. Fix version regex for Gazelle platform. |
!4107 | Fix Raisecom.ROS.get_lldp_neighbors script |
!4136 | noc/noc#1162 Raisecom.ROS profile. Fix convert iface name for SNMP ifDescr format. |
SKS.SKS
MR | Title |
---|---|
!3782 | SKS.SKS. Add SNMP support. |
!3807 | SKS.SKS. Fix convert interface name on LLDP. |
Ubiquiti.AirOS
MR | Title |
---|---|
!3028 | Add ubiquiti/wa support |
!4010 | Fix get_interface |
Zyxel.MSAN
MR | Title |
---|---|
!3906 | fix_profile_Zyxel_MSAN |
rare
MR | Title |
---|---|
!3785 | Alstec.MSPU.get_interfaces. Fix ‘port ’ command. |
!3806 | Qtech.QSW8200. Add SNMP support. |
!3808 | DCN.DCWL.get_metrics. Fix CPU Usage. |
!3820 | Qtech.QSW8200.get_version. Fix bootprom detect on SNMP. |
!3844 | Update Dlink.DxS_Smart profile to support DGS-1210 rev. F1 series |
!3858 | Fix Eltex.LTP.get_version script |
!3859 | Fix Huawei.MA5300.get_interfaces script |
!3861 | Add get_fqdn scripts. |
!3924 | Rotek.RTBSv1.get_interfaces. Fix subinterfaces. |
!3931 | Fix Eltex.LTP.get_mac_address_table script |
!3952 | Iskratel.MBAN.get_version. Fix regex for SHDSL board. |
!3958 | TPLink.T2600G. Add SNMP support. |
!3959 | Vector.Lamda. Add SNMP Support. |
!3961 | Fix Proscend.SHDSL.get_chassis_id script |
!4023 | Fix DLink.DxS_Cisco_CLI.get_lldp_neighbors script |
!4062 | Fix DLink.DxS_Cisco_CLI.get_lldp_neighbors script |
!4078 | Fix Planet.WGSD.get_interfaces script |
!4093 | Fix ZTE.ZXA10.get_interfaces script |
!4095 | Fix Eltex.DSLAM profile |
!4108 | Fix Eltex.LTE.get_version script |
!4206 | ConfDB normalizer Raisecom.RCIOS + tests. |