Josh-CO Dev

Solving the worlds problems one line of code at a time.

1 Comment

Automating a WebInspect Scan using PowerShell and the WebInspect API

I have been struggling with the WebInspect API for quite some time now. It seems that I could always call the get methods just fine, but some of the post methods had some weird syntax that I could never figure out. After a lot of trial and error, I finally figured out how to use PowerShell to start a scan with the WebInspect API. I am posting this here hoping it will help out others. It seems that Google does not turn up anything on the API other than some HP help files which really do not help.

$wiapiScan = "http://vmwebinspect01:8083/webinspect/scanner" #set this to the location of your webinspect API instance

$body = @{
    settingsName="Default"    #the settings name to use for this scan
        ScanName="Testing"    #The name to give the scan
        StartUrl="http://scrubbed"   #the url to scan
        CrawlAuditMode="CrawlOnly"   #crawl, audit, or both.
        StartOption="Url"    #refer to the API documentation for the other options
    } | convertto-json

$response = Invoke-webrequest -Method Post -Body $body -Uri $wiapiScan #put it all together

Be sure that $wiapiScan is set to point to your WebInspect instance. Remember that the API has to be configured and started wherever you are hosting WebInspect. In my case, I am using a dedicated VM.

Leave a comment

PowerShell – Browser Automation

I have had a need to perform browser automation for some time. Apparently, this can be done pretty easily with PowerShell, providing you have a basic understanding of the DOM and how to manipulate it.

Start by create an Internet Explorer object. I know, I know. Internet Explorer. PowerShell is Microsoft, so just get over it. We all know it sucks.

#create a new browser object to handle all of our navigation
    $Browser = New-Object -com InternetExplorer.Application

If you want to watch the automation, which you do because it’s cool, be sure to set the visible property to true.

#go ahead and show the browser window
    $Browser.visible = $true

To browse to a specific url, just use the navigate method.

#Navigate to the URL Defined above

In some cases, you need to tell the thread to sleep for an arbitrary amount of time. Otherwise the script will just keep running and you won’t accomplish much. I put a statement similar to this after almost every call.

#Sleep the thread while the page is loading
    Start-Sleep -Milliseconds 1000;

To click on an item, you need to know it’s name in the DOM


You can set the value of a text-box

$Browser.Document.getElementById("tbName").value = $reportName

Check a box

$Browser.Document.getElementById("CheckBoxList1_1").checked = $true

There’s a lot more that can be done. Play around with the Browser.Document properties a little bit. In the end, it’s a lot easier to use a tool like Auto IT, but it’s cool to script out some automation.

Leave a comment

PowerShell – Delete folders older than a certain timeframe based on a filename

I was recently asked to come up with a script that will go through a directory and delete any folders based on a naming convention. The naming convention was something like 20150730…….The obvious pattern here is YYYYMMDD……. Now that we know this, a loop and some substrings are our friend.

$directory = "directory to purge"

Get-ChildItem $directory | Where-Object {$_.PSIsContainer} | ForEach-Object {
    $DirName = $_.Name

    $DirYear = $_.Name.Substring(0,4)
    $DirMonth = $_.Name.Substring(4,2)
    $stringDate = $DirMonth + "/01/" + $DirYear

    $FolderDTime = [datetime]$stringDate
    $Today = get-date

    $timespan = NEW-TIMESPAN –Start $Today –End $FolderDTime

    if ($timespan.Days -lt -365) {
        Remove-Item $_.FullName
        write-output ((get-date).ToShortDateString() + " - " + $DirName + " was deleted") | out-file -append "C:\LogRhythmArchives\Inactive\del_archives_log.txt"


So, let’s break this down.

Get-ChildItem $directory | Where-Object {$_.PSIsContainer} | ForEach-Object {

This is our for loop. This does a Get-ChildItem of $directory. Be sure to set this variable to the folder you want to loop through. The where object is a weird powershell way of saying, exclude files and only give me directories.

    $DirName = $_.Name
    $DirYear = $_.Name.Substring(0,4)
    $DirMonth = $_.Name.Substring(4,2)
    $stringDate = $DirMonth + "/01/" + $DirYear

First, take our goofy name and throw it into a variable. Use a substring to pull the first four characters to make the year, then similarly for the month. Now we use concatenation to build a fake date.

$FolderDTime = [datetime]$stringDate

$Today = get-date

    $timespan = NEW-TIMESPAN –Start $Today –End $FolderDTime

Now, we can take our fake date string and tell powershell to force it into a date time object. Now that we have this, we can create a timespan object to determine the length of time between the file name and today.

    if ($timespan.Days -lt -365) {
        Remove-Item $_.FullName
        write-output ((get-date).ToShortDateString() + " - " + $DirName + " was deleted") | out-file -append "del_archives_log.txt"

Finally, we check to see if the timespan is greater than 365 days. They way I did the timespan creation, we get left with a negative number so we check less than. Now we can call remove-item to permanently delete the file, and write our output to a log.

Please note, remove-item does not send the items to the recycle bin. It is a permanent deletion.

Leave a comment

Powershell – Extract Data from a Database

Another handy thing to do with powershell is extract data from a database. Like everything else Powershell, this is incredibly easy to do.

First you need to setup and connect to the database itself.

	#create our database connection
	$SqlConnection = New-Object System.Data.SqlClient.SqlConnection
	$SqlConnection.ConnectionString = "Server=ServerName;Database=DatabaseName;Integrated Security=True"

This should be pretty straight forward. Just replace servername and databasename with your information. This connection string is for sqlserver 2012, you can substitute in the connection string for any sql server database.

Next we need to set up our command and hook it up.

	#create and set up our sql command
	$SqlCmd = New-Object System.Data.SqlClient.SqlCommand

        $SqlCmd.CommandText = "select * from dbo.table"
	$SqlCmd.Connection = $SqlConnection
        $SqlCmd.CommandTimeout = 0

It’s worth noting here that setting the commandtimeout to 0 overrides the default timeout, which I was having trouble with.

Next, just call the execute reader and load the results in a table.

	#execute our command
	$result = $SqlCmd.ExecuteReader()
	#load the results of our command into a datatable
	$table = new-object System.Data.DataTable

If you have ever programmed in .NET before, all of this should look very familiar. Now, you can do whatever you like with the data. In my case, I loop through it and assign the values to a variable so I can use them later.

	foreach($row in $table)
		#set up our variables based on the data row
		$EmailAddress = $row.Item("Email")

Also, don’t forget to close out your connection after you are done. This will not happen automatically.

        #close out our sql connection to prevent it from staying open. 

Leave a comment

Powershell – Scrubbing Data From a Webpage

One function that I have found a lot of use for is to write a script to go out to a website and scrub out data. With traditional programming languages, this was always a little tricky without calling a webservice or something like that to process the request. Powershell makes this surprisingly easy, especially if you are familiar with HTML and know what to look for. I am attaching a commented script below that should give you the basics of how to scrub a site for data. In this case, I am hitting the NIST NVE and passing in a CVE as a parameter to get more information.

#This function is responsible for connecting to the NIST NVE and retrieving the CVE data from the database.
#$Link - the url to the cve entry 
function GetCVEDataFromNIST($Link)
	#assign our parameter to a local variable
	$url = $Link
	#create a new webrequest to go out and fetch the html contents
	$result = Invoke-WebRequest $url
	#$html = $result.ParsedHtml.getElementsByTagName("span") | Where "classname" -match "^label" | Select -ExpandProperty InnerText
	#parse the html contents to find 

tags with a class of row. The inner text will contain the CVE Summary $Summary = $result.ParsedHtml.getElementsByTagName("p") | Where "classname" -match "^row" | Select -ExpandProperty InnerText #parse the html contents to find

tags with a class of row. The inner text will contain the published date and cve score $Published = $result.ParsedHtml.getElementsByTagName("div") | Where "classname" -match "^row" | Select -ExpandProperty InnerText #Start the concatenation of data. This will take the summary and add a new line $AllCVEData = ("{0} `n" -f $Summary) #parse through the published and cve score and separate them out into two lines then concatenate into allcvedata foreach($element in $Published) { #write-host $element $AllCVEData += "`n {0}" -f $element } #write-host $AllCVEData #return the summary, published date, and cve score return $AllCVEData }

A couple of notes about this. First, if you work for a corporation, you probably need to go through a proxy. See my other article for that. Second, you have to know the html of the site. In the code above, we are looking for


elements that have a specific class and this tells us where our data is.

In this script, I am defining link in another function and passing it in to this one. For the curious minded, this is how I am building the url.

$Link = ("{0}&search_type=all&cves=on" -f $CVE)

Again, you have to research the site. In this case, the url takes a parameter in the query string of the cve. You can simply sub this out with your own input and get custom results back. For the evil-minded out there, please don’t try to fuzz the inputs on NIST’s site, it likely won’t end well for you.

Leave a comment

Powershell – Sending an Email

Another task that I am commonly automating in Powershell is the sending of emails. As long as you have some SMTP credentials, this is relatively easy. Please note that I have removed any sensitive information.

#this function is responsible for all mail handling
#$to - the email address of the recipient
#$subject - the subject of the email
#$body - the body of the email
function sendMail($to, $subject, $body)
     Write-Host "Sending Email"

     #SMTP server name
     $smtpServer = ""     

     #Creating a Mail object
     $msg = new-object Net.Mail.MailMessage

     #Creating SMTP server object
     $smtp = new-object Net.Mail.SmtpClient($smtpServer)

     #Email structure
	 #all emails will come from the XXXXXX mailbox. 
     $msg.From = ""
     $msg.ReplyTo = ""
     $msg.subject = $subject
     $msg.body = $body

     #send the mail

Leave a comment

Powershell – Set a Proxy

I’ve been using Powershell A LOT lately and I thought I would start sharing some code snippets of things that I have had to learn along the way. The first thing is how to set up powershell to use a proxy. A lot of the scripts I have been writing involve going out to the web and scrubbing HTML as a data source. It is pretty simple. Just add this to your script (I always put it at the top out of habit, not sure if this is necessary).

#Set our proxy information. We are hardcoding the proxy server, but the credentials will pull from the OS
$global:PSDefaultParameterValues = @{