As part of my ongoing effort to migrate everything over to Python 3, it's time to show this "quick example" in Python 3.
TextFSM is a powerful parsing tool (python module) developed by Google. There are some great examples out there to get you started. Here are two I urge you to read if this topic is of interest to you:
- TextFSM and Structured Data By Kirk Byers
- Programmatic Access to CLI Devices with TextFSM from Jason Edelmans blog
I can never get enough of examples so here is a very simple one to get you started or keep you practicing. I find that a quick example where I can see results of my own making really energizes my learning process.
For this example, you need python 3 and the textfsm module installed.
TextFSM Getting Started is an excellent resource which includes the installation process for textfsm (textfsm is a pip installable module).
In addition to the environment, you will need 2 things to get started.
- A TextFMS template
- Content to parse with the TextFMS template
The TextFSM Pattern Matching or "Parsing" Template
The template may be the tricky part. This template defines the structure of the data you are trying to gather or parse out of your content. We are very fortunate that the Network to Code (NTC) team has given us a large library of templates from which to choose and so very often we don't have to worry too much about the template itself. We just need to know what Cisco IOS show command has the information we want.
Content (network device output) to Parse
Once you have your template, you need content to parse. You can get this in a variety of ways. You can query your devices real time via Ansible or via a python script or you can act on file based (saved) data that you already have.
In this example we will keep it simple and assume we have a text file of show commands that we need to parse to get the device hardware information and software version.
To get hardware and software information, the "show version" output should have what we want. So looking at the existing templates in the NTC library, it looks like the cisco_ios_show_version.template has what we need, If we look at the template we can see that it has two variables, VERSION and HARDWARE (which will return a list). That looks just about right for the information we want to extract and luckily the file of show commands includes the output of the "show version" command.
So here are the two files we will work with in this example:
- the textFSM template file (downloadable from GitHub) and included my textfsm3 repository on GitHub.
ntc-templates/templates/cisco_ios_show_version.template
- the content file with show commands including the output of the show version command (downloadable here)
lab-swtich-show-cmds
For simplicity I've put them both in a temp directory and I will launch the python interpreter from there so we can work real time. I also list the modules that I have in the virtual environment. The only one you need for this example is textfsm.
Working directory and files
(txtfsm3) Eugenias-PB:textfsm3 eugenia$ tree
.
├── cisco_ios_show_version.template
└── lab-swtich-show-cmds.txt
The environment
(textfsm3) Eugenias-PB:~ eugenia$ python --version
Python 3.6.55
(textfsm3) Eugenias-PB:textfsm-example eugenia$ pip freeze
ciscoconfparse==1.3.20
colorama==0.3.9
dnspython==1.15.0
dnspython3==1.15.0
ipaddress==1.0.22
textfsm==0.4.1
PB:textfsm-example eugenia$
Lets get started...
Launch the interpreter and import the textfsm module. The ">>>" tells you that you are in the python command interpreter.
(textfsm) Eugenias-PB:textfsm3 eugenia$ python
Python 3.6.5 (default, Apr 25 2018, 14:26:36)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
Open the template file into a file handle I've called template and pass that as the argument to the textfsm.TextFSM method creating a "parsing object" based on that template (the show version template in our case).
>>> template = open('cisco_ios_show_version.template')
>>> results_template = textfsm.TextFSM(template)
Look at some of the methods (functions) available in the results_template object by using dir() on the results_template object you just created. Make note of the 'ParseText' one as that is the one we will use shortly to parse our content.
>>> import textfsm
>>> template = open('cisco_ios_show_version.template')
>>> results_template = textfsm.TextFSM(template)
>>> dir(results_template)
['GetValuesByAttrib', 'MAX_NAME_LEN', 'ParseText', 'Reset', '_AppendRecord', '_AssignVar', '_CheckLine', '_CheckRule', '_ClearAllRecord', '_ClearRecord', '_DEFAULT_OPTIONS', '_GetHeader', '_GetValue', '_Operations', '_Parse', '_ParseFSMState', '_ParseFSMVariables', '_ValidateFSM', '_ValidateOptions', 'class', 'delattr', 'dict', 'dir', 'doc', 'eq', 'format', 'ge', 'getattribute', 'gt', 'hash', 'init', 'init_subclass', 'le', 'lt', 'module', 'ne', 'new', 'reduce', 'reduce_ex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook', 'weakref', '_cur_state', '_cur_state_name', '_line_num', '_options_cls', '_result', 'comment_regex', 'header', 'state_list', 'state_name_re', 'states', 'value_map', 'values']
>>> results_template.value_map
{'VERSION': '(?P.+?)', 'ROMMON': '(?P\S+)', 'HOSTNAME': '(?P\S+)', 'UPTIME': '(?P.+)', 'RELOAD_REASON': '(?P.+?)', 'RUNNING_IMAGE': '(?P\S+)', 'HARDWARE': '(?P\S+\d\S+)', 'SERIAL': '(?P\S+)', 'CONFIG_REGISTER': '(?P\S+)', 'MAC': '(?P[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5})'}
>>>
You don't need these commands. I just wanted to show you how to investigate the object a little which is quite easy when working in the interpreter. Start with dir() which will give you the various methods or actions you can perform on the object you created. Focus on the ones without the underscore as those are generally internal methods although in many cases they can be useful. In the case of the textfms module I use the _results and the header method often.
This is the heart of what we are trying to do now that we've selected our parsing template and created an object that will parse our content and pull out the information we want. So far all we have done is built a specific "strainer" to catch the information we need. How we need to pour our data through our strainer.
First we open our text file of show commands. Below I create a file handle (basically a variable) to the show command file called content2parse and read the contents of our text file of show commands into the variable. You have to open the file first before you can read its contents.
>>> content2parse = open('lab-swtich-show-cmds.txt')
>>> content = content2parse.read()
Now we parse out the data we want using our results_template object and its ParseText method against our content and store the results in the parsed_results variable. As you can see, this is a list of lists. If you ran this against show command files from 5 different devices you can begin to see the possibilities. You would have a list of 5 lists with the show version information for each device.
>>> parsed_results = results_template.ParseText(content)
>>> parsed_results
[['15.2(2)E3', 'Bootstrap', 'artic-sw01', '4 days, 14 hours, 2 minutes', 'c2960s-universalk9-mz.152-2.E3.bin', ['WS-C2960S-24TS-S'], ['FOC1709W1DT'], '0xF']]
On a side note, notice that the value of results_template._result is equal to our parsed_results. Its a good practice to put your results in
>>> results_template._result
[['15.2(2)E3', 'Bootstrap', 'artic-sw01', '4 days, 14 hours, 2 minutes', 'Reload command', 'c2960s-universalk9-mz.152-2.E3.bin', ['WS-C2960S-24TS-S'], ['FOC1709W1DT'], '0xF', ['70:10:5C:53:D4:80']]]
Your first inclination might be to iterate over the results but remember that its a list of lists (notice the double opening square brackets, [[, the first denoting that the entire result is a list and the second denoting that the first element in the list is also a list) so iterating over the results would iterate over only the item in the "top level" list [0]. What you want is to iterate over each element of the results list which is also a list. I know that my top level list has 1 element, the [0] element, and that list has 8 elements. We only parsed one file which is why our list only has one element. If I iterate over the [0] element of the list I get each individual bit of information. Get the length of the top level list (should only have one element since we only parsed 1 file)
Get the length of list (the "outer" or top level list):
>>> len(parsed_results)
1
>>> for item in parsed_results:
>>> … print(item)
>>> …
['15.2(2)E3', 'Bootstrap', 'artic-sw01', '4 days, 14 hours, 2 minutes', 'Reload command', 'c2960s-universalk9-mz.152-2.E3.bin', ['WS-C2960S-24TS-S'], ['FOC1709W1DT'], '0xF', ['70:10:5C:53:D4:80']]
Next, get the length of that first element (the 0 element as lists are zero indexed). Here is our data for our one device. If we iterate over our outer list we get a single line of output with the one element (also a list).
Lets find out how many elements our inner list has:
>>> len(parsed_results[0])
10
>>> index = 0
>>> for item in parsed_results[0]:
... print(f"Element # {index}: \t{item}")
... index += 1
...
Element # 0: 15.2(2)E3
Element # 1: Bootstrap
Element # 2: artic-sw01
Element # 3: 4 days, 14 hours, 2 minutes
Element # 4: Reload command
Element # 5: c2960s-universalk9-mz.152-2.E3.bin
Element # 6: ['WS-C2960S-24TS-S']
Element # 7: ['FOC1709W1DT']
Element # 8: 0xF
Element # 9: ['70:10:5C:53:D4:80']
>>>
I know I have 10 elements (0-9) because i checked the length and if you do a length on the value_map you can see that the template is designed to get 10 pieces of information so that makes sense. Get the length of the first list
>>> len(parsed_results[0])
10
>>> len(results_template.value_map)
10
Finally, after all of this I wanted to get the version and hardware information out of this file and here it is:
>>> parsed_results[0][0]
'15.2(2)E3'
>>> parsed_results[0][6]
['WS-C2960S-24TS-S']
>>>
I hope these basics help you get started or keep practicing.
Here are all the main commands in sequence so you can see the flow without all of the interruptions. Also easier to copy and paste!
>>> import textfsm
>>> template = open('cisco_ios_show_version.template')
>>> results_template = textfsm.TextFSM(template)
>>> content2parse = open('lab-swtich-show-cmds.txt')
>>> content = content2parse.read()
>>> parsed_results = results_template.ParseText(content)
>>> parsed_results
[['15.2(2)E3', 'Bootstrap', 'artic-sw01', '4 days, 14 hours, 2 minutes', 'Reload command', 'c2960s-universalk9-mz.152-2.E3.bin', ['WS-C2960S-24TS-S'], ['FOC1709W1DT'], '0xF', ['70:10:5C:53:D4:80']]]