Need help with collecting configs and writing to file in nornir

[edited]
Hi,
I am trying to backup configs from an arista and junos device.
however, running this code results in write file being written with “Incomplete command”.
it looks like the for loop runs successfully first time, but it never ends and then netmiko forwards empty_command and code completes. appreciate any help or pointers.
thanks in advance!

from nornir import InitNornir
from nornir.plugins.tasks.files import write_file
import pathlib
import datetime as dt
from nornir.plugins.tasks.networking import netmiko_send_command
from pprint import pprint
from nornir.plugins.tasks.networking import napalm_get
from nornir.plugins.functions.text import print_result

nr = InitNornir()

junosd = nr.filter(platform="junos")

def collect_config_j(task):
  for dev in junosd.inventory.hosts.items():
    config_dir = "configs"
    entry_dir = config_dir + "/" + str(dev[1])
    pathlib.Path(config_dir).mkdir(exist_ok=True)
    pathlib.Path(entry_dir).mkdir(exist_ok=True)
    import ipdb; ipdb.set_trace()
    jout = task.run(task=netmiko_send_command, command_string="show configuration")
    task.run(task=write_file, content=jout.result, filename=f"" + str(entry_dir) + "/" + str(dt.datetime.today().strftime('%Y-%m-%d')) + ".txt")

def main():
  mj=nr.run(task=collect_config_j, num_workers=1)

main()

I am unable to understand why would it enter the loop again here :confused:
the script completed one run for all devices in the host.yaml and the write_file has the correct info, but it enters loop again for some reason…


ipdb> n
> /home/test/test.py(29)collect_config_j()
     28 def collect_config_j(task):
---> 29   for dev in junosd.inventory.hosts.items():
     30     config_dir = "configs"

ipdb> n
--Return--
None
> /home/test/test.py(29)collect_config_j()
     28 def collect_config_j(task):
---> 29   for dev in junosd.inventory.hosts.items():
     30    

You are executing that loop for every single device. Remember that the goal of nornir is to dispatch each task to each device already, so you are basically running each task for each device in the nr object which in turns will run that for loop around junosd.

Try replacing mj=nr.run(task=collect_config_j, num_workers=1) with mj=junosd.run(task=collect_config_j, num_workers=1)

And removing the for loop in the task. In the task you will have task.host with the host information.

1 Like

Thanks much, David!
I have amended to the following, this works but am not sure if this is the optimal way :slight_smile:

junosd = nr.filter(platform="junos")
eosd = nr.filter(platform="eos")

def collect_config_j(task):
    config_dir = "configs"
    entry_dir = config_dir + "/" + task.host.name
    pathlib.Path(config_dir).mkdir(exist_ok=True)
    pathlib.Path(entry_dir).mkdir(exist_ok=True)
    jout = task.run(task=netmiko_send_command, command_string="show configuration")
    task.run(task=write_file, content=jout.result, filename=f"" + str(entry_dir) + "/" + str(dt.datetime.today().strftime('%Y-%m-%d')) + ".txt")

def collect_config_e(task):
    config_dir = "configs"
    entry_dir = config_dir + "/" + task.host.name
    pathlib.Path(config_dir).mkdir(exist_ok=True)
    pathlib.Path(entry_dir).mkdir(exist_ok=True)
    jout = task.run(task=netmiko_send_command, command_string="show running-config")
    task.run(task=write_file, content=jout.result, filename=f"" + str(entry_dir) + "/" + str(dt.datetime.today().strftime('%Y-%m-%d')) + ".txt")

def main():
  mj=junosd.run(task=collect_config_j, num_workers=1)
  rj=eosd.run(task=collect_config_e, num_workers=1)

main()

Yes, looks good. I’d suggest two ways of making it simpler though:

  1. Using napalm. Although that would require netconf on the junos side
  2. Instead of having two tasks having just one and looking at the platform. Something like:
def collect_config(task):
    ...
    commands = {
         "junos": "show configuration",
         "eos": "show running-config",
    } 
    out = task.run(
              task=netmiko_send_command,
              command_string=commands[task.host.platform]
    )
    ...
1 Like

awesome, thanks a mil!
I was unable to get napalm_get working on arista as it was not being run properly. hence failed back to netmiko_send_command which seems to do the job for now.
Wish to continue exploring this amazing tool you’ve built.

You might want to check out this toolkit I built which solves your issue. Also, I’ve done the heavy lifting to collect all possible NAPALM getters as a bonus too :call_me_hand:

how to write_file only if changed : True ?

I have some code to repeat failed hosts or if file zero

# Check if any fail and zero then tryagin
cntOut=0
while(result.failed==True and cntOut <= 5):
    tryaginList = list(nr.data.failed_hosts)
    files = []
    # r=root, d=directories, f = files
    for r, d, f in os.walk(path):
        for file in f:
            if '.txt' in file:
                files.append(os.path.join(r, file))

    # print(len(files))
    # for f in files:
    #     print(f)

    for i in range(len(files)):

        if os.stat(files[i]).st_size == 0:
    #         print(files[i])
            fileName = os.path.basename(files[i])
            fileName = os.path.splitext(fileName)[0]
            Router = fileName.split('-')
            print(Router[0])
            if Router[0] not in tryaginList:
                tryaginList.append(Router[0])
                
    tryaginList = list(dict.fromkeys(tryaginList))  
#     print(tryaginList)
    nr = InitNornir(
        core={"num_workers": num_workers},
        inventory={
            "plugin": "nornir.plugins.inventory.simple.SimpleInventory",
            "options": {
                "host_file": host_file,
                "group_file": group_file
            }
        }
    )
    tryagin = nr.filter(F(hostname__any=tryaginList))
    result = tryagin.run(task=collect_config)
#     print_result(result)
    nr.close_connections()
#     print(nr.data.failed_hosts)
    cntOut=cntOut+1

how to write_file only if changed : True ?

This issue is probably lost here because it’s an unrelated question. Open a new topic if you still need help here.