frr: T6746: additional improvements after 10.2 upgrade (#4259)

* smoketest: T6746: add substring search in getFRRconfig()

Some daemons (e.g. bgpd) have several nested substrings/sections like

router bgp 100
 address-family ipv4 unicast
 ..
 exit-address-family
exit

We can now use getFRRconfig() with the substring option to extract only
 address-family ipv4 unicast
 ..
 exit-address-family

Making config validation more granular

* frrender: T6746: only re-render FRR config if config_dict did change

* frrender: T6746: fix naming glitch isis/eigrp

* frrender: T6746: add --stdout option when running with debug flags

* smoketest: T6746: remove unneeded commit_guard time

It was an invalid workarround as the underlaying issue seems to be a race
condition in CStore.

The commit process is not finished until all pending files from
VYATTA_CHANGES_ONLY_DIR are copied to VYATTA_ACTIVE_CONFIGURATION_DIR. This is
done inside libvyatta-cfg1 and the FUSE UnionFS part. On large non-interactive
commits FUSE UnionFS might not replicate the real state in time, leading to
errors when querying the working and effective configuration.

TO BE DELETED AFTER SWITCH TO IN MEMORY CONFIG

1 files changed, 29 insertions, 19 deletions

diff --git a/smoketest/scripts/cli/base_vyostest_shim.py b/smoketest/scripts/cli/base_vyostest_shim.py
index 9cf6a653a..2be25ff22 100644
--- a/smoketest/scripts/cli/base_vyostest_shim.py
+++ b/smoketest/scripts/cli/base_vyostest_shim.py
@@ -18,7 +18,6 @@ import paramiko
 import pprint
 
 from time import sleep
-from time import time
 from typing import Type
 
 from vyos.configsession import ConfigSession
@@ -27,10 +26,17 @@ from vyos import ConfigError
 from vyos.defaults import commit_lock
 from vyos.utils.process import cmd
 from vyos.utils.process import run
-from vyos.utils.process import process_named_running
 
 save_config = '/tmp/vyos-smoketest-save'
 
+# The commit process is not finished until all pending files from
+# VYATTA_CHANGES_ONLY_DIR are copied to VYATTA_ACTIVE_CONFIGURATION_DIR. This
+# is done inside libvyatta-cfg1 and the FUSE UnionFS part. On large non-
+# interactive commits FUSE UnionFS might not replicate the real state in time,
+# leading to errors when querying the working and effective configuration.
+# TO BE DELETED AFTER SWITCH TO IN MEMORY CONFIG
+CSTORE_GUARD_TIME = 4
+
 # This class acts as shim between individual Smoketests developed for VyOS and
 # the Python UnitTest framework. Before every test is loaded, we dump the current
 # system configuration and reload it after the test - despite the test results.
@@ -45,7 +51,6 @@ class VyOSUnitTestSHIM:
         # trigger the certain failure condition.
         # Use "self.debug = True" in derived classes setUp() method
         debug = False
-        commit_guard = time()
         @classmethod
         def setUpClass(cls):
             cls._session = ConfigSession(os.getpid())
@@ -86,14 +91,12 @@ class VyOSUnitTestSHIM:
             if self.debug:
                 print('commit')
             self._session.commit()
-            # during a commit there is a process opening commit_lock, and run() returns 0
+            # During a commit there is a process opening commit_lock, and run()
+            # returns 0
             while run(f'sudo lsof -nP {commit_lock}') == 0:
                 sleep(0.250)
-            # wait for FRR reload to be complete
-            while process_named_running('frr-reload.py'):
-                sleep(0.250)
-            # reset getFRRconfig() guard timer
-            self.commit_guard = time()
+            # Wait for CStore completion for fast non-interactive commits
+            sleep(CSTORE_GUARD_TIME)
 
         def op_mode(self, path : list) -> None:
             """
@@ -108,20 +111,27 @@ class VyOSUnitTestSHIM:
                 pprint.pprint(out)
             return out
 
-        def getFRRconfig(self, string=None, end='$', endsection='^!', daemon='', guard_time=10, empty_retry=0):
-            """ Retrieve current "running configuration" from FRR """
-            # Sometimes FRR needs some time after reloading the configuration to
-            # appear in vtysh. This is a workaround addiung a 10 second guard timer
-            # between the last cli_commit() and the first read of FRR config via vtysh
-            while (time() - self.commit_guard) < guard_time:
-                sleep(0.250) # wait 250 milliseconds
-            command = f'vtysh -c "show run {daemon} no-header"'
-            if string: command += f' | sed -n "/^{string}{end}/,/{endsection}/p"'
+        def getFRRconfig(self, string=None, end='$', endsection='^!',
+                         substring=None, endsubsection=None, empty_retry=0):
+            """
+            Retrieve current "running configuration" from FRR
+
+            string:        search for a specific start string in the configuration
+            end:           end of the section to search for (line ending)
+            endsection:    end of the configuration
+            substring:     search section under the result found by string
+            endsubsection: end of the subsection (usually something with "exit")
+            """
+            command = f'vtysh -c "show run no-header"'
+            if string:
+                command += f' | sed -n "/^{string}{end}/,/{endsection}/p"'
+                if substring and endsubsection:
+                    command += f' | sed -n "/^{substring}/,/{endsubsection}/p"'
             out = cmd(command)
             if self.debug:
                 print(f'\n\ncommand "{command}" returned:\n')
                 pprint.pprint(out)
-            if empty_retry:
+            if empty_retry > 0:
                 retry_count = 0
                 while not out and retry_count < empty_retry:
                     if self.debug and retry_count % 10 == 0: