This post is english translation of my article for czech Technet Flash Magazine.
The main reason why Microsoft started working on PowerShell was to give administrators better tool for their tasks than they had. With PowerShell they can automate their daily tasks very easily. PowerShell proved that it is really amazing tools and that's why Microsoft integrated PowerShell to Windows Serve 2008, Windows 7 or SQL Server 2008. There are other libraries for IIS, Exchange, VMware management and many more others.
So, when PowerShell is so great why only administrators should use it? Why not programmers or advanced users? If you have created any .bat file or were scriptinng in old command line, Powershell is the right choice for you. It's syntax is much more comfortable and related to common programming languages. Besides that there are very interesting commands (cmdlets) integrated into the core.
As you guess this article is written for programmers. It should point out some tips there could make their work easier.
A simple console application problems
First of all I will emphasize the main reason why PowerShell is so popular (among programmers) – tasks that you normally solve with a single-purpose command line C# application can be solved in PowerShell very easily. What does it mean "very easily"? You only need to create a ps1 file and then run it in PowerShell console. (of course using script is not necessary, you can write the commands directly into the console)
How we did it without PowerShell? In Visual Studio we created a C# command line project and add some code into generated Main method. We needed to parse arguments again, browse the directory tree manually as in our previous project again, open/read files, process errors and exceptions etc. Then build the project and run the exe file. It is so tedious for one simple task and there are lots of files (source, project, etc.).
C# is great for complex processing where we can use all its nature. If you compare it to PowerShell there is significant code noise. Generally speaking if the task can be done in .NET, then it can be done in PowerShell. The typical task might be: go through all the files in given directory, open them and replace string XYZ with ABC.
Text replacement was just an example. Some time ago I saw someone who was trying to deal with duplicate pictures. He had many pictures from his camera and some of them were duplicated in more directories. He wanted to find the duplicates and ignore Exif because some of them were already edited. If you are interested in the resulting script, just have a look at stack overflow.
As a very short example let's try to sort pictures by their orientation (vertical vs. horizontal) and sort the result by the orientation and name. The pictures can be anywhere in the directory structure:
$d = gci D:\temp\ dscn*.jpg -rec |
% {
$b = new-object System.Drawing.Bitmap $_.FullName
New-Object PsObject -prop @{
Orientation=if($b.Height -gt $b.Width) { 'v' } else {'h' };
Path=$_.fullname }
$b.Dispose()
} |
sort -Property Orientation,Path
The code could be even shorter but then it would not be readable.
Can you see how easily we can gain the results? Did I hear woow? ;)
You can find many similar problems in you daily life, just use your phantasy. In the following I'll cover some concrete howtos and PowerShell features.
Exploring libraries
Sometimes you have to deal with unknown assemblies (or COM) or you have only some examples that you would like to check. For
example I found a library that works with StackOverflow API. You don't know nothing about it, just some examples. So you download the zip and...
PS> Add-Type -Path c:\temp\StackOverflowApi\StackExchangeApi.dll
PS> $uri = new-object System.Uri 'http://code.google.com/p/stackexchange-api/'
PS> [StackExchangeApi.SXAPI]::Initialize('SXAPI Example Code', $uri)
We initialized the library as in the example and lets have a look at how we can get a user:
PS> [STackExchangeApi.User] | gm -static
TypeName: StackExchangeApi.User
Name MemberType Definition
---- ---------- ----------
Equals Method static bool Equals(System.Object objA, System.Object objB)
GetUsersForName Method static System.Collections.Generic.ICollection`1[[StackExcha...
GetUserWithId Method static StackExchangeApi.User GetUserWithId(long id, StackEx...
ReferenceEquals Method static bool ReferenceEquals(System.Object objA, System.Obje...
PS> [STackExchangeApi.User]::GetUsersForName
MemberType : Method
OverloadDefinitions : {....GetUsersForName(string name, StackExchangeApi.StackExchangeSite site)}
...
Ok, that means that there is something new - StackExchangeApi.StackExchangeSite. What is it?
PS> [STackExchangeApi.StackExchangeSite] | gm -sta
TypeName: StackExchangeApi.StackExchangeSite
Name MemberType Definition
---- ---------- ----------
...
ToObject Method static System.Object ToObject(type enumType, System.Object v...
Meta Property static StackExchangeApi.StackExchangeSite Meta {get;}
ServerFault Property static StackExchangeApi.StackExchangeSite ServerFault {get;}
StackOverflow Property static StackExchangeApi.StackExchangeSite StackOverflow {get...
SuperUser Property static StackExchangeApi.StackExchangeSite SuperUser {get;}
Ok, we got the idea. It is the site where we want to look for the user. Let's try it.
PS> [STackExchangeApi.User]::GetUsersForName('skeet', 'StackOverflow')
Site : StackOverflow
Id : 22656
Badges : {Nice Answer, Nice Answer, Nice Answer, Nice Answer...}
RecentActivity : {2341601, 2341288, 2341035, 2340025...}
ReputationGraph : StackExchangeApi.ReputationGraph
Name : Jon Skeet
Answers : {7244, 9033, 131871, 137448...}
Questions : {194484, 215548, 236907, 247621...}
FavoriteQuestions : {212726, 282329, 348175, 406760...}
Gravatar : http://www.gravatar.com/avatar/6d8ebb117e8d83d74ea95fbdd0f87e13?s=50&d=ide...
Reputation : 141022
GoldBadgeCount : 29
SilverBadgeCount : 542
BronzeBadgeCount : 1073
# and 4 more users are returned
PowerShell is based on .NET so you can easily look at the properties, call the methods, even use events etc.
Do you know any quicker way how to experiment with unknown API?
The same holds for COM objects. Look at quick example how to work with Skype.
Working with XML
One of the feature that you will like is how PowerShell deals with XML. Imagine that you can work with XML like with an object – attributes become string properties and nested elements become object properties. Sounds familiar? Yes, it is similar to (de)serialization. But when deserializing a new object of some type is created. When working with XML from PowerShell we get XmlDocument which is very well known in .NET world.
It's time for an example. Let's create a new test file.
PS> @"
<root>
<article date="2010-12-01">
<name>Discover new dimensions</name>
<body>Discover them now. Go!</body>
</article>
<article date="2000-01-01">
<name>Future</name>
<body>what will be with us in ten years?</body>
</article>
</root>
"@ | Set-Content c:\temp\flashtest.xml
We will read the file and cast it to xml (this is how accelerators work) and look at the result.
PS>$x = [xml](gc c:\temp\flashtest.xml)
PS>$node = $x.root.article | Where-Object { [datetime]$_.date -lt [datetime]'2005-01-01' }
PS>$node
date name body
---- ---- ----
2000-01-01 Future what will be with us in ten years?
PS>$node.name = 'Near ' + $node.name
PS>$x.root.article[0].date = (get-date).ToString('yyyy-MM-dd')
Variable $x is of type XmlDocument, whereas $node of type XmlElement. If you want to set a new value of an attribute or of an element string, just use simple assignment. If you need to add a new element, then we have to switch to .NET methods.
PS>$note = $x.CreateElement('note')
PS>$note.InnerText = 'poor content'
PS>$node.AppendChild($note)
PS>$x.root.article[1]
date name body note
---- ---- ---- ----
2000-01-01 Near Future what will be with us in ten years? poor content
Sometimes you would like to use XPath. In V1 you needed to do it using some .NET objects. In V2 it is much more easier, we were given cmdlet Select-Xml.
PS>Select-Xml -Xml $x -XPath '//article[contains(body/text(),"ten")]'
PS>
PS>$ns = @{ e = "http://www.w3.org/1999/xhtml" }
PS>Select-Xml -Path $somePath -Xpath //e:div[@id] -names $ns
If you wonder what can be used as a source for Select-Xml, just have a look at Get-Help Select-Xml. You will see three options: xml object(s) itself (as in the example above), path to file(s) or xml(s) as a string.
After you are done and you want to save the result, you have to use .NET method again:
PS>$x.Save('c:\temp\res.xml')
Regex tester
PowerShell is very handy when there is a need to test regular expressions. I won't describe what regular expressions are and what they solve. You can find more info at Regular-expressions.info.
I'll show you how to check that the regular expression does what expected.
Checking the regex in PowerShell is the quickest way if you have PowerShell console open (note: if you lost the console among the other windows, have a look at AutoHotkey). It is much more lengthy to test regexes on web online or run special programs. There are several possibilities.
Operator -match
This is the simplest approach. Left operand is (array of) string(s) and right operand is the regular expression.
PS>'jeho email je karel@novak.cz' -match ' \w+@[a-zA-Z_]+\.(?<d>[a-zA-Z]{2,3})'
True
PS>$matches
Name Value
---- -----
d cz
0 karel@novak.cz
PowerShell uses collection $matches with regex groups from last evaluation with -match operator. Group 0 contains all the matching string. Regex is not case sensitive. In fact it corresponds to regular expression without any special options.
In case you will need case sensitivity, use operator -cmatch.
Operator -match can be also used as a filter. Open your console and try it, you will figure it out immediatelly:
PS>'1a','2b','1c' -match '1\w'
PS>'1a' -match '1\w'
Accelerator [regex]
You can use [regex] to create a regular expression object:
PS>$r = [regex]'^\w+@[a-zA-Z_]+\.(?<d>[a-zA-Z]{2,3})$'
PS>$r.GetType().FullName
System.Text.RegularExpressions.Regex
It creates instance of class Regex (well known for .NET programators) and saves it to variable $r. Again there are no special options. Let's have a look on the options:
PS>$r | fl Options
You can work with the object as you know from .NET. In case you don't remember some signatures, you may have a look at the members:
PS>$r | gm
TypeName: System.Text.RegularExpressions.Regex
Name MemberType Definition
---- ---------- ----------
...
GetGroupNames Method string[] GetGroupNames()
GetGroupNumbers Method int[] GetGroupNumbers()
…
IsMatch Method bool IsMatch(string input), bool IsMatch(strin
Match Method System.Text.RegularExpressions.Match Match(st
Matches Method System.Text.RegularExpressions.MatchCollection
Replace Method string Replace(string input, string replacemen
...
Object creation
The last option, that is closely related to the previous one, is to create regex using cmdlet New-Object. Then you can use any constructor of Regex class – e.g. specify options like multiline, singleline etc.
PS>$opts = [System.Text.RegularExpressions.RegexOptions]'MultiLine,SingleLine'
PS>$r = new-object Text.RegularExpressions '^\w+@[a-zA-Z_]+?\.(?<d>[a-zA-Z]{2,3})$',$opts
Look at the interesting syntax sugar how to specify regex options – I haven't found it documented anywhere. I found it accidentally at Oisin's blog
Using Enumerated types (Enums) in PowerShell.
Generally it's a joy to work with enums in PowerShell. You don't need to specify type of the enum, just its name is enough.
Furthermore – try this:
PS>[string]::Compare('a','a',[Globalization.CultureInfo]::CurrentCulture, 'test')
Cannot convert argument "3", with value: "test"..... The possible enumeration values are "None,
IgnoreCase, IgnoreNonSpace, ... Ordinal"."
PS>[string]::Compare('a','a',[Globalization.CultureInfo]::CurrentCulture,'IgnoreCase')
PowerShell will suggest you valid values – so you don't need to look into documentation! This will speed up your productivity if you work with enums.
Note #1: You can of course just use static method of class [regex]:
PS>[regex]::IsMatch('karel@novak.cz', '^\w+@[a-zA-Z_]+\.(?<d>[a-zA-Z]{2,3})$')
Note #2: You know operator -match. There is another very useful operator -replace, that is used for text replacement. Here is just a quick and simple illustration:
PS>(Get-Content c:\test.txt) -replace 'abc','def' | Set-Content c:\test.txt
Function FindR
Later in section about clipboard I use FindR. It's a tiny filter that returns data that match given regular expression.
filter FindR($regex) {
[Regex]::Matches($_, $regex, 'IgnoreCase') | % { $_.Value }
}
You can pipe almost anything to FindR. PowerShell will try to convert it to string because method Matches expects string as its parameter. If we pipe objects, PowerShell runtime decides how to convert the objects to string.
For example dir | findr '.*gs.*' will work, but it will return only names of files and directories (that match the regex).
But Get-WinEvent -LogName Application -MaxEvents 10 | findr '.*instal.*' will not work, because PowerShell will pass string 'System.Diagnostics.Eventing.Reader.EventLogRecord' to method Matches. This string is probably just a result of ToString() called on objects from Get-WinEvent.
You are able to come up with some realistic scenarios for FindR, I'm sure. I use it when I need to parse logs from our customers. In the last case I had some file sizes in the log and I needed to find out average size and total sum of the sizes. To parse concrete sizes I used look behind.
gc c:\dev\WO\wpdataimport.log |
findr -r '(?<=Job content file size: )\d+' |
Measure-Object -Sum -Average
First command reads the file, the second one finds only lines that contain 'Job content file size' and returns only numbers that follow the string and the last command does the statistics. Quick and easy. This, Jane, this is PowerShell :)
Encoding, decoding, conversions …
Especially web developers need to work with base64 strings or encode/decode urls. You can do it of course via some online tools or specialized programs. But again, it is much more easier to add this functions to your profile and the conversions will be available all the time.
Add-Type -AssemblyName System.Web
function FromBase64([string]$str) {
[text.encoding]::utf8.getstring([convert]::FromBase64String($str))
}
function ToBase64([string]$str) {
[convert]::ToBase64String([text.encoding]::utf8.getBytes($str))
}
function UrlDecode([string]$url) {
[Web.Httputility]::UrlDecode($url)
}
function UrlEncode([string]$url) {
[Web.Httputility]::UrlEncode($url)
}
function HtmlDecode([string]$url) {
[Web.Httputility]::HtmlDecode($url)
}
function HtmlEncode([string]$url) {
[Web.Httputility]::HtmlEncode($url)
}
Admins working solely in command line might use this function that helps with searching on Google:
PS> function Run-GoogleQuery {
Start-Process ('http://www.google.cz/search?q=' + (UrlEncode ($args -join " ")))
}
PS> Set-Alias qg Run-GoogleQuery
PS> qg this is test
I used HtmlEncode when I was working on this article and I needed to insert PowerShell code into <pre ..> tag.
With function clip (you will find it later) it was very easy:
HtmlEncode (clip) | clip
Conversion Html2Xml
Sometimes you will need to convert html to xml. Regexes? No! I hope you have heard that parsing HTML with regexes is very tricky. Instead, you may use free library SgmlReader.
function Convert-Html2Xml {
param(
[Parameter(ValueFromPipeline=$true)][object[]]$html
)
begin { $sb = new-object Text.StringBuilder(20kb) }
process { $html | % { $null = $sb.AppendLine($_) } }
end {
# no default namespace, thx.
$str = $sb.ToString().Replace(' xmlns="http://www.w3.org/1999/xhtml"', '')
Add-Type -Path G:\bin\SgmlReaderDll.dll
$sr = new-object io.stringreader $str
$sgml = new-object Sgml.SgmlReader
$sgml.DocType = 'HTML';
$sgml.WhitespaceHandling = 'All';
$sgml.CaseFolding = 'ToLower';
$sgml.InputStream = $sr;
$xml = new-object Xml.XmlDocument;
$xml.PreserveWhitespace = $true;
$xml.XmlResolver = $null;
$xml.Load($sgml);
$sgml.Close()
$sr.Close()
$xml
}
}
There are two ways how to work with the function:
PS>$x1 = gc c:\temp\testhtmlsource.delete.html | Convert-Html2Xml
PS>$x1.Save('c:\temp\test1.xml')
PS>$x3 = Convert-Html2Xml (gc c:\temp\testhtmlsource.html)
PS>$x3.Save('c:\temp\test2.xml')
You can use the conversion in cases where given web site doesn't have public API and you need to work with it only through its WEB UI (nightmare!). One example could be translator http://www.slovnik.cz. Complete solution (quick & dirty) uses XPath via cmdlet Select-Xml.
Some other quick & dirty code? I built up one for my colleague who creates from time to time links to movies database CSFD manually. Don't look for beautiful code, you will find only automation.
And last quick example – you found a blog with many posts and all the posts contain full text. It can look like this. Rss feed doesn't help because it returns only some of them. You would like to see headings of the posts because you are just curious. Obviously you can scroll down and catch the headings. But what about this:
$xml = Convert-Html2Xml (download-page 'http://www.nivot.org/CategoryView,category,PowerShell.aspx' )
Select-Xml -Xml $xml -XPath '//div[@class="itemTitle"]/a/text()' | % { $_.Node.Value }
PowerShell 2.0 - About Dynamic Parameters
PowerShell 2.0 – Introducing the PModem File Transfer Protocol
PowerShell 2.0 - Enabling Remoting with Virtual XP Mode on Windows 7
PowerShell 2.0 goes RTM for ALL Platforms
PowerShell 2.0 - Module Initializers
PowerShell 2.0 – Getting and setting text to and from the clipboard
PowerShell 2.0 – Asynchronous Callbacks from .NET
PowerShell – Function Parameters & .NET Attributes
PowerShell 2.0: A Configurable and Flexible Script Logger Module
....
Clipboard
For the first time you will wonder: "Clipboard? Why should I care?". I use it very often as a transport mechanism between an application and PowerShell. You have seen several times that I use clip. One more sample will follow:
I have a problem with deadlocks in sql server. I run sql server from console with switches for deadlock detections. After sql server prints some info about resolved deadlocks, I select the text in console, copy it and go to PowerShell. I would like to find out the SPIDs.
(clip) | FindR -r 'SPID: \d+' | select -unique
This will return IDs of all processes involved.
Take it just as an example, I know that I could use SQL analyzer.
And how the functions for clipboard management look like?
Add-Type –a system.windows.forms
function Set-ClipBoard {
param(
[Parameter(Mandatory=$true,ValueFromPipeline=$true,Position=0)][object]$s
)
begin { $sb = new-object Text.StringBuilder }
process {
$s | % {
if ($sb.Length -gt 0) { $null = $sb.AppendLine(); }
$null = $sb.Append($_)
}
}
end { [windows.forms.clipboard]::SetText($sb.Tostring()) }
}
function Get-ClipBoard {
[windows.forms.clipboard]::GetText()
}
# clip works as Get-Clipboard and Set-Clipboard; it depends on the context:
# gc c:\test.txt | clip
# clip | sc c:\test.txt
function clip {
param(
[Parameter(Mandatory=$false,ValueFromPipeline=$true)][object]$s
)
begin { $sb = new-object Text.StringBuilder }
process {
$s | % {
if ($sb.Length -gt 0) { $null = $sb.AppendLine(); }
$null = $sb.Append($_)
}
}
end {
if ($sb.Length -gt 0) { $sb.Tostring() | Set-ClipBoard}
else { Get-ClipBoard }
}
}
Note that clipboard management is available only if PowerShell is run with -STA switch or in ISE environment. It's because it uses WinForms in the background.
In case you need to have clipboard available in MTA (default mode), look at PowerShell 2.0 – Getting and setting text to and from the clipboard.
Is there something missing?
Yes, of course. There are some more topics that deserve its own article. There is a great cmdlet New-WebServiceProxy, you can work with SQL server (using .NET classes), you can create GUIs (WinForms or WPF), use remoting (for deployment and continuous integration), … and much more. Hopefully I will cover some of them next time.
You have seen basic features that I find very attractive. However, it's up to you if you will became fan of PowerShell or if you think that the features are not worth learning. But remember, that the most important reason why you should use PowerShell is the first part – quick and effective solution of common tasks. That's where PowerShell is excellent.