At my current assignment I am working purely in a DevOps role. The biggest chunk of my day-to-day work is the migration of an on-premise TFS 2013 instance to on-premise TFS 2015, including process template changes, migration to the new build eco-system (so moving away from XAML templates) and introducing new functionality as SonarQube etc. As the new build-system is fairly new, there is not that much information on the web yet about writing your own build steps, so I thought it could be helpful to others as well if I wrote down my findings.

For all publishing activities of my custom build-steps, I use the TFS CLI which can be downloaded here: https://github.com/Microsoft/tfs-cli.

The basics of custom buildsteps
A build-step in TFS2015 technically consists of a single folder with a manifest (task.json) which you use to configure the properties, an executable Powershell script that performs the actual task during the build and an icon file that is displayed in the list of build-steps for your definition. To make it easy for you, a command is available to generate a blueprint with you need at once:

tfx build tasks create --auth-type basic

It all starts with defining the task: you need to specify the form fields needed to configure the task, by adding textboxes, checkboxes, file- and folder-pickers and picklists (selectboxes). Each form-field can have a help-text which is a handy feature to assist the user to fill the fields with the appropriate values. Make sure to use a unique id for each of your tasks (GUID) and after each change you make to the task.json file, update the version number to make sure each of the buildagents will update their local copies of the task. If you re-use the same versionnumber, local copies of the script are not being updated, hence strange behaviour will occur. Here’s the code for the example I will use in the rest of this post:

{
	"id": "CD6A4B69-CE6A-4A09-A866-FEA612B80702",
	"name": "RobinPaardekamDebug",
	"friendlyName": "Robin Paardekam Debug Task",
	"description": "Expand variables that are available during the build and try all kinds of stuff locally.",
	"author": "Robin Paardekam, Studio 010 - Digitale Media",
	"helpMarkDown": "RobinPaardekamDebug v0.0.1. This buildstep will write all environment variables and their values so you can easily find the one that you are looking for. There are also some other controls on this page, but they are just intended for demo-purposes. Nothing is mandatory on this task.",
	"category": "Utility",
	"visibility": [
		"Build",
		"Release"
	],
	"demands": [ ],
	"version": {
		"Major": "0",
		"Minor": "0",
		"Patch": "1"
	},
	"groups": [
	{
		"name": "Dynamic",
		"displayName": "Dynamic Lists (demo)",
		"isExpanded": false
	}],
	"minimumAgentVersion": "1.95.0",
	"instanceNameFormat": "Display all build environment variables",
	"inputs": [
	{
		"name": "connectedServiceTfs",
		"type": "connectedService:Generic",
		"label": "TFS Service Endpoint",
		"required": false,
		"helpMarkDown": "Create an endpoint to TFS if you want to populate the pickLists in the Dynamic Lists section. E.g. for Test: http://tfstest/tfs"
	},
	{
		"name": "TfsProject",
		"type": "pickList",
		"label": "TFS Project",
		"defaultValue": "",
		"required": false,
		"groupName": "Dynamic",
		"helpMarkDown": "This list is populated with Projectnames from the TFS REST API. You are allowed to enter another value as well.",
		"properties": {
			"EditableOptions": "True"
		}
	},
	{
		"name": "TfsProjectTeam",
		"type": "pickList",
		"label": "TFS Project.Team",
		"defaultValue": "",
		"required": false,
		"groupName": "Dynamic",
		"helpMarkDown": "This list is populated with Teamnames for the selected Team Project from the TFS REST API. You are not allowed to enter another value.",
		"properties": {
			"EditableOptions": "False"
		}
	},
	{
		"name": "TfsProjectBuilddef",
		"type": "pickList",
		"label": "TFS Project.BuildDefs",
		"defaultValue": "",
		"required": false,
		"groupName": "Dynamic",
		"helpMarkDown": "This list is populated with Build Definitions for the selected Team Project from the TFS REST API. You are not allowed to enter another value.",
		"properties": {
			"EditableOptions": "False"
		}
	}],
	"sourceDefinitions": [
	{
		"target": "TfsProject",
		"endpoint": "http://tfsserver/tfs/DefaultCollection/_apis/projects#$(connectedServiceTfs)",
		"selector": "jsonpath:$.value[*].name",
		"authKey": "$(connectedServiceTfs)"
	},
	{
		"target": "TfsProjectTeam",
		"endpoint": "https://tfsserver/tfs/DefaultCollection/_apis/projects/$(TfsProject)/teams#$(connectedServiceTfs)",
		"selector": "jsonpath:$.value[*].name",
		"authKey": "$(connectedServiceTfs)"
	},
	{
		"target": "TfsProjectBuilddef",
		"endpoint": "https://tfsserver/tfs/DefaultCollection/$(TfsProject)/_apis/build/definitions#$(connectedServiceTfs)",
		"selector": "jsonpath:$.value[*].name",
		"authKey": "$(connectedServiceTfs)"
	}],
	"execution":
	{
		"PowerShell": {
			"target": "$(currentDirectory)\\executeTask.ps1",
			"argumentFormat": "",
			"workingDirectory": "$(currentDirectory)"
		}
	}
}

As you can see in the final block of the manifest, a PowerShell file is defined that will run when this task is started. Now you should start writing the actual logic for the task, so you will need to receive all input from the build-step and start processing it in your own piece of code. At my client we chose to only use PowerShell but you are also allowed to use NodeJS if you plan to target other platforms then Windows. One thing to be aware of: each of the input fields that are defined in the task manifest should be defined as ‘input parameters’ for the executable script. The task will fail when it is being run when one of the fields is not received by the build step itself. For this specific example, I used the following PowerShell script:

[cmdletbinding()]
param (
    [string]$cwd,
    [string]$connectedServiceTfs,
    [string]$TfsProject,
    [string]$TfsProjectTeam,
    [string]$TfsProjectBuilddef
)

Write-Verbose "Importing modules"
import-module "Microsoft.TeamFoundation.DistributedTask.Task.Internal"
import-module "Microsoft.TeamFoundation.DistributedTask.Task.Common"

Write-Host "INPUT (User) workingDir: $cwd"
Write-Host "INPUT (User) TfsProject: $TfsProject"
Write-Host "INPUT (User) TfsProjectTeam: $TfsProjectTeam"
Write-Host "INPUT (User) TfsProjectBuilddef: $TfsProjectBuilddef"

$environmentVars = get-childitem -path env:*

foreach($var in $environmentVars)
{
 $keyname = $var.Key
 $keyvalue = $var.Value
 
 Write-Output "${keyname}: $keyvalue"
}

Write-Host ("##vso[task.complete result=Succeeded;]DONE")

exit 0

The script above will display all server-variables that are available at build-time, which is a neat little helper when you are working on writing new tasks. The pickList values from the task are not actually used, their values are just being displayed as this is just a demo.

Passing variables between build steps
Once you have created one or more buildsteps for your business-needs, you will likely run into the situation that (string-)output from one step is needed as input in a following step. When I worked on TFS 2013 build workflows this wasn’t an issue at all: you create a variable in the correct scope and it is available for you to consume it. However: a task runs in its own context so the regular PowerShell variables are not persistent through out the rest of the build-process. There are some tricks to fix that: using VSO Logging Commands, I found the command below, which enabled me to access the parameter in a next buildstep.

Write-Host "##vso[task.setvariable variable=VariableName;]VariableValue"

In a following step you can access this variable using the statement in a way as illustrated below:

Write-Host "The following variable was found: $(VariableName)"

The above and many more task logging commands are documented here. Another cool command I hope to discuss on short term is the “task.addattachment” which can be used to add specific content on the build output front-page (aka. timeline).

Consuming data from REST APIs
One of the coolest things I have done with my custom build-steps so far is getting data from external sources to be displayed in the picklists of my build-steps. E.g. I have some configuration values that are stored in a custom made application that are needed to define how the build-step behaves, or another example (used in the script above) is to fill the pickLists with values that are coming from TFS itself. Let’s look a bit closer on how to achieve this:

The “sourceDefinitions” block in the manifest file defines where to get the data from. However: they are dependent on the Service Endpoint defined as the first inputfield, called “connectedServiceTfs”. This control will show all Service Endpoints configured for the current Team Project (unfortunately there is not an easy way to share Service Endpoint between Team Projects at this moment). If you haven’t configured any Service Endpoints yet (the pickList filters for Generic Endpoints only) you can do so using the manage-link at the righthand side.

As illustrated in the image above, you need to name the endpoint (so you can easily distinguish them when your list grows in the future) and you need to add a service address, username and password. As this blogpost is only addressing on-premise TFS 2015, this is all you will need. For VSTS you’ll need to do more manual preparations to use Personal Access Tokens etc but that is out of scope for this particular example. In this case, the main thing is to save the credentials so the task-page can easily connect once you define your build.

Now that you have defined your Generic Service Endpoint, the task should be able to populate the fields defined in your manifest: TfsProject, TfsProjectTeam and TfsProjectBuilddef. When I started writing the task I assumed that I could simply request the Service address from the selected endpoint, avoiding the fixed paths in my manifest. Unfortunately I did not manage to get the endpoint address, probably for security purposes. For now I can live with the hardcoded paths, although it would be great to dynamically change the source of the pickList, based on the endpoint. If you can come up with a fix for this, I am looking forward to any response!

The task contains 1 additional groupbox for the dynamic pickLists. Once you have selected a Team Project from the first list, the other 2 boxes can also be populated accordingly. Using the Webdeveloper tools in Internet Explorer or Chrome (press F12) you can troubleshoot any issues when your lists remain empty. As mentioned earlier: there is no real use in this task currently for these 3 pickList values, but I think it shows how easily you can consume a REST service from your task page.

As you see in the selector-property in the manifest, you have to define an jsonpath for selecting the appropriate values. If you’ve got experience with XML and XPath this shouldn’t be too much of a problem. It might help to use a site like JsonPath.com to debug your specific rest-calls.

Adding information to buildoutput frontpage
As mentioned earlier in this post, another cool VSO Command is the “task.addattachment”. It allows you to
upload and attach summary markdown to current timeline record. It took me some trial-on-error to get this working, but eventually I got it working as expected. The following snippet will create a local text-file with the information I want to display on the build-frontpage and upload and attach it to the timeline.

### Add additional information to the timeline.
  $logFilePath = "{0}\BuildDemo_Front_{1}.txt" -f ($Env:BUILD_BINARIESDIRECTORY, $Env:BUILD_BUILDID)
  $text = "This is just some not-important content to show how to add Log-output to the build-dashboard. Make it useful!"
  $text | Set-Content "$logFilePath"
  Write-Host "##vso[task.addattachment type=Distributedtask.Core.Summary;name=Demo Output On Timeline;]$logFilePath"
### End additional information on timeline script

The snippet above will add a new caption on your timeline called “Demo Output On Timeline” followed by the text that you’ve declared in the $text-variable. It can be useful for adding build-specific URLs, network-paths etc. so your users won’t have to dig through the (verbose-)logs but can easily click the links on the timeline. Note that you can use markdown-syntax here!

Uploading your new task to TFS
Once you have all your files ready, you can publish the task to your TFS-application server. Note that you will need to enable Basic Authentication on the tfs Virtual Directory in IIS as that is currently the only way you can talk to the tfs-API when running TFS2015 on-premise. The following command will start publication of your task (first answer the three questions “Address of your TFS instance”, “Username”, “Password”) to the application server, making your build-step available when editing a definition. Just to be clear: I am NOT addressing any XAML-build stuff in this blogpost. Everything in this post is about the new build ecosystem.

tfx build tasks upload --auth-type basic --task-path ./FolderWithYourBuildStep

The TFS Rest Api documentation can be found here. Update September 2016: I wrote another blog post regarding TFS 2015 build and release tasks, mainly focussing on the TFS REST API.