Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download from Source Control #30

Open
jessehouwing opened this issue Apr 7, 2016 · 12 comments
Open

Download from Source Control #30

jessehouwing opened this issue Apr 7, 2016 · 12 comments

Comments

@jessehouwing
Copy link
Owner

Download the specified file or folder(s) from source control to the specified location.

This can be especially powerful for Release, allowing the ability to pull scripts or tools from source control without having to make them an artifact.

@peder
Copy link

peder commented Jun 24, 2016

We have old XAML build scripts that prompt the developer to choose which branch to publish. As far as I can tell, the new build system does not provide this functionality without perhaps extensive PowerShell scripting. Do you anticipate that a build script could have a variable like $branchPath, steps to download the project's source, and then normal artifact-producing build steps?

@jessehouwing
Copy link
Owner Author

jessehouwing commented Jun 24, 2016 via email

@peder
Copy link

peder commented Jun 24, 2016

In a nutshell, we need the ability to change which branch we're building from, just like Git repositories have:

image

@peder
Copy link

peder commented Jun 24, 2016

As far as CI goes, I haven't seen how teams manage to combine CI+TFVC+Feature Branching without creating new build scripts for each feature branch.

@jessehouwing
Copy link
Owner Author

The only way I imagine this could work is to either download additional folders to the workspace (as far as you can speak of a workspace, or offer the ability to create additional workspace mappings into which variables are expanded.

While these might work I expect a lot of issues around people trying to use this in CI/Gated/Shelveset builds and I suspect it will cause lots of issues with future branch policies if they ever make their way to TFVC.

Instead you can already achieve this functionality with a variable that you can set from the Queue window and a workspace with "build clean" turned off. as long as developers don't often queue old changeset versions, that should work just fine.

You could create a very simple task to either fail the build if the branch variable isn't set, or set it to a default value using my VSTS Variable Toolkit extension.

@peder
Copy link

peder commented Oct 4, 2016

Hey @jessehouwing, have you made any progress on downloading from source control? We're trying to build this out ourselves but running into module- and assembly-loading nightmares in our homegrown PowerShell scripts.

@danielstefanescu
Copy link

Hi @jessehouwing,
Any news regarding the download feature?

@peder
Copy link

peder commented Dec 19, 2017

So we ended up writing a script that we call using the PowerShell Task. It downloads files for a selected branch in a new workspace, undoing any local changes before getting latest. It then sets a build variable, Tenaska.SourcesDirectory, that can be used downstream in any build task. We allow the developer to set the $branchPath when they kick off a build by creating a "Settable at queue time" build variable.

Param (
     $agentHomeDirectory,
     $agentBuildDirectory,
     $collectionUri,
     $oauthToken,
     $sourceVersion,
     $workspaceName,
     $branchPath,
     $deleteWorkspace
)

# Set working variables

Write-Output "Download-from-any-branch working variables:"

Write-Output "agentHomeDirectory: $agentHomeDirectory"
Write-Output "branchPath: $branchPath"

$tfPath = $agentHomeDirectory + "\externals\vstshost\tf.exe"

$branchPathFolderStructure = $branchPath.Replace("$", "").Replace("/", "\")

$agentBuildDirectoryDrive = (Get-Item $agentBuildDirectory).PSDrive.Name + ":"
$localFolder = $agentBuildDirectoryDrive + "\src\" + $workspaceName + $branchPathFolderStructure
$serverFolder = $branchPath

$tfWorkspaceCountArgs = "vc", "workspaces", $workspaceName, "/computer:$ENV:COMPUTERNAME", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfCreateWorkspaceArgs = "vc", "workspace", "/new", "/location:local", "/permission:Public", $workspaceName, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfDeleteWorkspaceArgs = "vc", "workspace", "/delete", "/collection:$collectionUri", $workspaceName, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfMapFolderArgs = "vc", "workfold", "/map", "/workspace:$workspaceName",  $serverFolder, $localFolder, "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
#$tfGetLatestArgs = "vc", "get", "/version:$sourceVersion", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
# removed /version flag which was not compatible with multiple branches
$tfUndoAllArgs = "vc", "undo", "/recursive", "*", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
$tfGetLatestArgs = "vc", "get", "/recursive", "/overwrite", $localFolder, "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"
#$tfResetWorkspacesArgs = "vc", "workspaces", "/remove:*", "/collection:$collectionUri", "/loginType:OAuth", "/login:.,$oauthToken", "/noprompt"

$tfWorkspaceCountArgsString = [string]::Join(" ", $tfWorkspaceCountArgs)
$tfCreateWorkspaceArgsString = [string]::Join(" ", $tfCreateWorkspaceArgs)
$tfMapFolderArgsString = [string]::Join(" ", $tfMapFolderArgs)
$tfGetLatestArgsString = [string]::Join(" ", $tfGetLatestArgs)

Write-Output "tfWorkspaceCountArgs: $tfWorkspaceCountArgsString"
Write-Output "tfCreateWorkspaceArgs: $tfCreateWorkspaceArgsString"
Write-Output "tfMapFolderArgs: $tfMapFolderArgsString"
Write-Output "tfGetLatestArgs: $tfGetLatestArgsString"


# Now create the folder, map it to TFS, and emit a SourcesDirectory variable

Write-Output "Attempting to create local directory"
New-Item -ItemType Directory -Force -Path $localFolder | Out-Null

Write-Output "Changing working directory"
Set-Location $localFolder

Write-Output "Determine if workspace exists"
& $tfPath $tfWorkspaceCountArgs
$workspaceMeasure = & $tfPath $tfWorkspaceCountArgs | Measure-Object
$workspaceCount = $workspaceMeasure.Count - 3

Write-Output "Number of workspaces with name $workspaceName`: $workspaceCount"

if($workspaceCount -gt 0) {
    Write-Output "Workspace already exists"

    if($deleteWorkspace -eq $TRUE) {
        Write-Output "Deleting workspace"
        & $tfPath $tfDeleteWorkspaceArgs
        Write-Output "Creating workspace"
        & $tfPath $tfCreateWorkspaceArgs
    }
}
else {
    Write-Output "Creating workspace"
    & $tfPath $tfCreateWorkspaceArgs
}

Write-Output "Mapping folder to workspace"
& $tfPath $tfMapFolderArgs

Write-Output "Undoing any changed files"
& $tfPath $tfUndoAllArgs 2>&1 | Out-Null

Write-Output "Getting latest"
& $tfPath $tfGetLatestArgs

Write-Output "Setting Tenaska.SourcesDirectory variable and changing working location"
Set-Location $localFolder
Write-Output ("##vso[task.setvariable variable=Build.SourcesDirectory;]$localFolder")
Write-Output ("##vso[task.setvariable variable=Tenaska.SourcesDirectory;]$localFolder")

With this in place, we now basically have feature parity with Git Builds. One key missing piece, though: since this is just a bolted-on piece of functionality, the TFS build summary page always shows the same branch, even if a user selected a different $branchPath. I am curious if we change the Build.SourceBranch variable, would TFS save that value back? I'm suspecting we'd actually have to go into the Build table in the SQL database, which obviously would require TFS instead of VSTS. And since this is just to hack in support for multiple branches out of the same build definition for TFVC, there's probably no sense in pushing this any further.

@jessehouwing
Copy link
Owner Author

jessehouwing commented Dec 19, 2017 via email

@peder
Copy link

peder commented Dec 19, 2017

I did try that, and I also researched the REST API, which doesn't expose the branch path in their PATCH method. I think the only method available is via UPDATE statements directly against the collection database.

@peder
Copy link

peder commented Dec 19, 2017

https://stackoverflow.com/questions/25858650/update-status-of-tfs-builds-with-powershell

The examples in this post are likely specific to XAML builds, but there may be a way with PowerShell to change data on a Build object, including the Branch Path. I'll take a look at that and get back to this thread in another 1.5 years 👍

@jessehouwing
Copy link
Owner Author

jessehouwing commented Dec 19, 2017 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants