Nothing wholly SQL Server-related this time but this is something that I have been meaning to write for ages.
While searching on the internet for content I gather loads of URLs that might be useful to me and not long after this I lose track of what content is to be found at each location. We could add some value to the URLS by keeping the page title with each one. So, step up PowerShell and the invoke-webrequest function.
Pass the function below a URL, or a set of URLs, and it will go and fetch each page title for you. You can use this to make a more complete reference section in your documentation, blog or simply as a reminder of what content the URL leads to.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
function Get-WebpageTitleFromURL{ <# .Synopsis Takes in URLs and returns the URL with the web page title .DESCRIPTION Use to complete reference sections of documents or to add value to lists of URLs .EXAMPLE $URLs = ( 'http://www.bbc.co.uk', 'http://www.sqlservercentral.com', 'simple-talk.com/blogs/author/13359-jonathan-allen/', 'www.bing.com') Get-WebpageTitleFromURL $URLs | ft -AutoSize .INPUTS URLs - a single of comma separated set of URLs .OUTPUTS table with Title and URL columns #> [CmdletBinding()] Param ( # Param1 help description [Parameter(Mandatory=$true, ValueFromPipeline=$true)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] $URLs, [Parameter(Mandatory=$False)]$SaveToFile=$true ) begin{ $dt = New-Object system.data.datatable "Web page references." $dc1 = New-Object System.Data.DataColumn "Title",([string]) $dc2 = New-Object System.Data.DataColumn "URL",([string]) $dt.Columns.Add($dc1) $dt.Columns.Add($dc2) $c=1 $t= $URLs.Count } process{ foreach ($url in $URLs){ "Checking URL $c of $t" | Write-Output try{ $row = $dt.NewRow() $hsg = Invoke-WebRequest($url) $row.Title = $hsg.ParsedHtml.Title $row.URL = $url $dt.Rows.Add($row) } catch{ $row.Title = "Unable to get title." $row.URL = $url $dt.Rows.Add($row) } $c++ cls } $dt } } |
Let me know if this helps you in the comments below.
Load comments