netcrawl package¶
Subpackages¶
Submodules¶
netcrawl.cli module¶
Created on Feb 28, 2017
@author: Wyko
-
netcrawl.cli.
connect
(handler=None, netmiko_platform=None, ip=None, cred=None, port=None)[source]¶ Starts a CLI session with a remote device.
Uses Netmiko to start a SSH or Telnet session with a target device. It will attempt to use SSH first, and if it fails it will try Telnet. For each connection method, it will attempt each credential specified in the cred argument (if specified) or the config.cc.credentials list otherwise.
Keyword Arguments: - cred (dict) – If supplied, this method will only use the specified credential. Uses the config.cc.credentials list otherwise.
- port (int) – If supplied, this method will connect only on this port
- ip (str) – The IP address to connect to
- netmiko_platform (str) – The platform of the device, in the Netmiko format
- handler (ConnectHandler) – A Netmiko-type handler to use. Currently using one of Netmiko.ConnectHandler, Netmiko.ssh_autodetect.SSHDetect. Uses Netmiko.ConnectHandler by default.
Returns: A dict containing:
- connection (ConnectHandler): A Netmiko ConnectHandler object with a successfully opened connection
- tcp_22 (bool): True if port 22 is open
- tcp_23 (bool): True if port 23 is open
- username (str): The first successful credential’s username
- password (str): The first successful credential’s password
- cred_type (str): The first successful credential’s type
Return type: dict
Raises: IOError
– If a connection could not be establishedAssertionError
– If error checking failed
netcrawl.config module¶
netcrawl.core module¶
-
netcrawl.core.
nmap_scan
(target, **kwargs)[source]¶ Ping each host in a given range one at a time. When a live host is found, add it to the pending hosts database.
Parameters: target (str) – An Nmap compatible target specifier as outlined in the Nmap documentation Keyword Arguments: **kwargs – Arguments to pass to netcrawl.io_sql.main_db
-
netcrawl.core.
print_report
()[source]¶ Prints a brief report of the state of the databases to the console
-
netcrawl.core.
recursive_scan
(**kwargs)[source]¶ Starts a Recursive Scan (-sR) run. This is the main scanning method for netcrawl.
1. If a
target
kwarg is given, add that seed device to the list of pending deivces, even if it was already visited.2. Create workers (subprocesses) to perform the scanning work, up to 16 per CPU core, or up to the
processes
kwarg per core if that kwarg was given.3. Query the Pending table in the Main database for pending devices.
4. Autodetect the Netmiko platform for each device if needed.
5. Inventory the device using
netcrawl.devices.base.NetworkDevice.process_device()
- Add each discovered device to the Inventory database
Keyword Arguments: - skip_named_duplicates (bool) –
If True, this will cause netcrawl to skip neighbors which have the same hostname as a device that was previously visited.
Note
While this can potentially save a lot of time when scanning devices, if multiple different devices share the same hostname, they will not be scanned!
- target (str) – The IP address of a seed device to add to the pending devices database
- netmiko_platform (str) – The Netmiko platform of the
target
device, if one was given. - processes (int) – The number of worker processes to create, multiplied by the CPU count
Note
If there are any remaining keyword arguments in
**kwargs
, they will be passed tonetcrawl.io_sql.main_db
andnetcrawl.io_sql.device_db
-
netcrawl.core.
single_scan
(target, netmiko_platform='unknown')[source]¶ Starts a Single Scan (-sS) run. This scan polls a single device and presents information about the device to the console. Useful for testing a connection, as well as getting a quick overview of the target.
Keyword Arguments: - target (str) – The network address of the device to scan
- netmiko_platform (str) – The Netmiko platform of the
target
device. If one is not given, it will attempt to autodetect the device type.
netcrawl.device_dispatcher module¶
Controls selection of proper class based on the device type.
Credit: Kirk Byers
-
netcrawl.device_dispatcher.
autodetect
(target)[source]¶ This method invokes Netmiko’s autodetect functionality to determine the correct device class, then returns that class as a netmiko_platform.
Parameters: target (String) – The hostname or IP address to connect to
Raises: TypeError
– Could not find an appropriate class to inheritIOError
– Could not connect to the device
Returns: - The netmiko_platform representation of the proper
device class.
Return type: String
netcrawl.io_sql module¶
-
class
netcrawl.io_sql.
device_db
(**kwargs)[source]¶ Bases:
netcrawl.io_sql.sql_database
-
add_device_nd
(_device)[source]¶ Appends a device to the database
Parameters: _device (network_device) – A single network_device Returns: False if write was unsuccessful Int: Index of the device that was added, if successful Return type: Boolean
-
exists
(device_id=None, unique_name=None, device_name=None)[source]¶ Checks whether a device record is present in the devices table. Tries each supplied identifier in order until a match is found, then returns the device_id of the found record.
Keyword Arguments: - device_id (int) – If not None, check the device_id column for a match
- unique_name (str) – If not None, check the unique_name column for a match
- device_name (str) – If not None, check the device_name column for a match
- Returns
- int: The device_id of the first match found bool: False if not found
-
get_device_record
(column, value)[source]¶ Get a device record based on a lookup column. ‘WHERE column = value’
Returns: psycopg2 dict object
-
process_duplicate_device
(device)[source]¶ Parent method for handling an existing device which needs to be updated.
- Determine if the
device
exists and, if so, get the device_id - Overwrite all entries in the device with the new device
- Set a new updated time for all dependent tables
- Delete any interfaces and serials which no longer exist
- Add any new interfaces and serials
- Add any new MAC addresses
- Update any newly non-existent MAC addresses
Parameters: device (NetworkDevice) – A network device object to check against for duplicates Returns: True if a duplicate was found and updated Return type: bool - Determine if the
-
set_dependents_as_updated
(device_id, cur=None)[source]¶ Sets the last touched time on all dependents of the given device_id to now
-
update_device_entry
(device, cur=None, device_id=None, unique_name=None)[source]¶ Overwrites all entries in the Devices table with a matching
device_id
orunique_name
with the information indevice
.Parameters: device (NetworkDevice) – The device to source updates from
Keyword Arguments: - cur (psycopg2.cursor) – Cursor object used to update the database
- device_id (int) – If not None, overwrites the row at this index
with
device
- unique_name (str) – If not None, overwrites any row with a matching
unique_name
field withdevice
.
Note
If both
device_id
andunique_name
are given, the method will update all entries that match either key.Raises: ValueError
– Nounique_name
ordevice_id
passed to the method
-
-
class
netcrawl.io_sql.
main_db
(**kwargs)[source]¶ Bases:
netcrawl.io_sql.sql_database
-
add_device_pending_neighbors
(_device=None, _list=None)[source]¶ Appends a device or a list of devices to the database
- Optional Args:
- _device (network_device): A single device _list (List): List of devices
Returns: True if write was successful, False otherwise. Return type: Boolean
-
add_visited_device_nd
(_device=None, _list=None, cur=None)[source]¶ Appends a device or a list of devices to the database
- Optional Args:
- _device (network_device): A single network_device _list (List): List of network_device objects
Returns: True if write was successful, False otherwise. Return type: Boolean
-
-
class
netcrawl.io_sql.
sql_database
(**kwargs)[source]¶ Bases:
object
A base class to facilitate SQL database operations.
Keyword Arguments: clean (bool) – If True, this causes all database tables to be dropped in order to start with a clean database.
Warning
Obviously, this is really dangerous.
-
count
(table, column='*', value=None, partial_value=None, distinct=False, cur=None)[source]¶ Counts the occurrences of the specified
column
in a giventable
.Parameters: table (str) – The table to search in
Keyword Arguments: - column (str) – The column to count
- distinct (bool) – If True, count only unique matches
- value (str) –
If not None, adds a where clause to the count in the format:
WHERE column = 'value'
- partial_value (str) –
If not None, adds a where clause which will match a partial string in the format:
WHERE column like '%partial_value1%'
Returns: The number of matches
Return type: int
-
create_database
(new_db)[source]¶ Creates a new database
Parameters: new_db (str) – Database name to create
-
database_exists
(db)[source]¶ Returns true is the specified database exists
Parameters: db (str) – A database name Returns: True if the database exists Return type: bool
-
delete_database
(dbname)[source]¶ Deletes a database
Returns: True if the database was created
Return type: bool
Raises: FileExistsError
– If the database to be deleted does not exist.IOError
– If the database could not be deleted and still exists after execution
-
execute_sql
(*args, proc=None, fetch=True)[source]¶ Executes a SQL snippet and optionally gets the results
Parameters: *args – The arguments to pass along to
pyscopg2.cursor.execute()
. Usually a string containing the SQL statement, and potentially a tuple of parameters.Keyword Arguments: - proc (str) – The name of the parent process, for logging purposes
- fetch (bool) – If True, fetches all results from the query
Returns: The results of
pyscopg2.cursor.fetchall()
Return type: tuple
-
execute_sql_gen
(*args, proc=None)[source]¶ Executes a SQL snippet and gets the results in a generator
Parameters: *args – The arguments to pass along to pyscopg2.cursor.execute()
. Usually a string containing the SQL statement, and potentially a tuple of parameters.Keyword Arguments: proc (str) – The name of the parent process, for logging purposes Returns: The results of pyscopg2.cursor.fetchall()
Return type: generator
-
netcrawl.util module¶
-
class
netcrawl.util.
benchmark
(name)[source]¶ Bases:
object
Context manager which times the surrounded code and prints the results to the console
-
netcrawl.util.
cidr_to_netmask
(cidr)[source]¶ Changes CIDR notation to subnet masks. I honestly have no idea how this works. I just added some error checking.
-
class
netcrawl.util.
cleanExit
[source]¶ Bases:
object
Context manager who’s only purpose is to cleanly exit when the code execution is interrupted by the user
-
netcrawl.util.
clean_ip
(ip)[source]¶ Removes all non-digit or period characters from the source string
-
netcrawl.util.
contains_mac_address
(mac)[source]¶ Simple boolean operator to determine if a string contains a mac anywhere within it.
-
netcrawl.util.
getCreds
()[source]¶ Get stored credentials using a the credentials module. Requests credentials via prompt otherwise.
Returns: [{username, password, cred_type}, ] If the username and password had to be requested, the list will only have one entry.
Return type: List of Dicts
-
netcrawl.util.
netmask_to_cidr
(netmask)[source]¶ Translates a netmask to a CIDR format
Parameters: netmask (str) – A netmask in four octet ip address format Returns: The CIDR representation of the netmask Return type: int
-
netcrawl.util.
network_ip
(ip, subnet)[source]¶ Returns the network IP address calculated from the given
ip
andsubnet
.
-
netcrawl.util.
parse_ip
(raw_input)[source]¶ Returns a list of strings containing each IP address matched in the input string.
-
netcrawl.util.
port_is_open
(port, address, timeout=5)[source]¶ Checks a socket to see if the specified
port
is open.Parameters: - port (int) – The numbered TCP port to check
- address (str) – The address of the host to check
Keyword Arguments: timeout (int) – The number of seconds to wait before timing out. Defaults to 5 seconds. Zero seconds disables timeout
Returns: True if the port is open
Return type: bool