Consolidating TFVC repositories to Azure DevOps Git

During the past months I have been working on a project for a client which had a main goal, that consisted of consolidating a lot of old TFVC repositories from different old and new TFS and VSTS versions. When I finished the project, the client had only one Azure DevOps environment with only Git repositories. In this article I would like to share my approach.

TFS2008 and TFS2013

The oldest repositories were TFVC in either TFS2008 or TFS2013 and had to be converted to Git repositories. In Azure DevOps you can import TFVC repositories into Git but there are some limitations in size and days of history (changesets). So, I searched the web for some other solutions and came across some older tools and projects that could help me import repositories into Git.

The problem here is that these toolseither didn’t support the older TFS and TFVC versions anymore or simply didn’t work that great due to complex branch architecture in all the repositories. Eventually I used the well-known tool git-tfs: https://github.com/git-tfs/git-tfs#introduction

So, no problem, you would think.Wrong! The support for TFS2008 was dropped in some version of the tool simply because the underlying architecture of TFS had changed a lot, and it was not possible to be backwards compatible anymore, and besides: who still uses TFS2008 these days,right?! Luckily all the older versions are kept as a reference and are downloadable from the GitHub page, so I downloaded the “latest” version which still had the TFS2008 assemblies and support.

As the documentation stated I had to install git-tfs, and I did so by using Chocolatey. This created a folder on my PC in C:\tools\gittfs. The install also added a PATH entry to my computer environment variable so the command “git tfs” could be used from any command or PowerShell prompt.

I created a simple PowerShell script that I executed for every repository, which created a new Git repository in Azure DevOps:

//Create an empty folder to put the source files in from TFVC
cd\
Remove-Item –path c:\temp –recurse -force
mkdir c:\temp

//Do the actual “clone” of the old TFVC repository path
cd c:\temp
git tfs clone “http://teamserverURL:8080” "$/COMPLETE_PATH_TO_SOURCEFOLDER"

//Upload the new Git repository to Azure DevOps
cd c:\temp\SOURCEFOLDERNAME
git remote add origin https://AZURE_DEVOPS_URL/PROJECTNAME/_git/REPONAME
git push -u origin --all

This was one hurdle I took. I converted a lot of repositories from TFVC to Git. But with one thing in mind: I didn’t include all the branches created in TFVC. I only included one direct “path” from the source repository. In this case not a big problem, but something to keep in mind.

I did something similar for the TFS2013, but first I had to use a newer version of the git-tfs tool. So, I changed the path of the previously used older C:\gittfs to C:\gitstfs_OLD because maybe I had to use the old one again if something went wrong along the way. And it did! So, I was happy to be able to switch between the old version of the tool and the newer one.

For TFS2013 repositories I use a similar script but with some small changes:

//Create an empty folder to put the source files in from TFVC
cd\
Remove-Item –path c:\temp –recurse -force
mkdir c:\temp

//Do the actual “clone” of the old TFVC repository of a specific branch
mkdir c:\temp\TFS2013
cd c:\temp\TFS2013
git tfs clone "http://teamserverURL:8080/tfs/COLLECTIONNAME" "$/COMPLETE_PATH_TO_SOURCEFOLDER" .

//Upload the new Git repository to Azure DevOps
git remote add origin https://AZURE_DEVOPS_URL/PROJECTNAME/_git/REPONAME
git push -u origin --all

Existing Azure DevOps Git repositories

Besides the old TFS2008 and TFS2013 repositories the organization was already using Azure DevOps mainly for work item management and some building and releasing of newer software products.

With the existing import functionality within Azure DevOps of importing Git repositories without limitations it was easy to “move” the repositories, with complete history and branches, from different projects to the one big project with all the new and old repositories together, existing side by side. This was a side goal to get all the repositories in one Azure DevOps project to be able to get some trace-ability between code changes and work items.

Combining Git repositories

The last step involved combining some Git repositories into new ones, because the number of repositories was to big to handle, and some code needed to be shared between products. After making a list of repositories which should be combined, which was a very hard task, I created the following script to combine two (or more) repositories into one new one.

//Create an empty folder to create a new repository in
cd\
Remove-Item –path c:\temp –recurse -force
mkdir c:\temp
cd c:\temp
mkdir c:\temp\NEWrepoNAME
cd c:\temp\NEWrepoNAME

//Initialize a new Git repository and do an initial commit because else you cannot //merge some other repository files into it, which will be done in the next step
git init
dir > deleteme.txt
git add .
git commit -m “Initial dummy commit”
git rm .\deleteme.txt
git commit -m “Clean up initial file”

//Clone the first old repository and move to a subfolder in the new repository
git remote add -f repoNAME1 https://AzuerDevOpsURL/PROJECTNAME/git/repoNAME1git merge repoNAME1/master --allow-unrelated-histories
mkdir repoNAME1_Migrateddir –exclude repoNAME1_Migrated | %{git mv $.Name repoNAME1_Migrated}
git commit -m “Move repoNAME1 files into subdir”
git remote remove repoNAME1

//Clone the second old repository and move into subfolder in the new repository
git remote add -f repoNAME2 https://AzuerDevOpsURL/PROJECTNAME/git/repoNAME2git merge repoNAME2/master --allow-unrelated-histories
mkdir repoNAME2_Migrateddir –exclude repoNAME2_Migrated, repoNAME1_Migrated | %{git mv $.Name repoNAME2_Migrated}
git commit -m “Move repoNAME2 files into subdir”
git remote remove repoNAME2

//Optional
Optionally: manually insert a .gitignore file in de root folder
Optionally: Rename migrated subfolders to original folder names

//Do the commit with some sensible comment
git add .
git commit -m “Combined repoNAME1 and repoNAME2 into NEWrepoNAME”

//Add an origin to the newly created combined repository and push changes
git remote add origin https://AzuerDevOpsURL/PROJECTNAME/_git/NEWrepoNAME
git push -u origin --all

Things to keep in mind here:

  • The new repository NEWrepoNAME needs to be created first, and be empty, in Azure DevOps.
  • We move the old repositories into subfolders of the new repository.
  • We do a merge of two (or more) repositories, therefore history viewing is more complex. An extra click is needed to view the history before the rename/move into subfolder action.
  • With the second (and third and fourth) repository you want to merge into the new one, you must exclude all the previous ones when moving to a subfolder.
  • We use different names (repoName2_Migrated) for the subfolders because if in the folder a child folder with the same name exists (in a lot of cases it will!) the moving of files will not move all the right files. The subfolder names can be renamed, after the move, to the original folder names again.
  • Don’t forget to delete the original two or more repositories you combined to prevent users making changes in the “old” repositories.

Feel free to contact me if this article was helpful of if you have questions or remarks.

8 Replies to “Consolidating TFVC repositories to Azure DevOps Git”

  1. Hello,

    I am having problems to migrate a DevOps TFVC repository to DevOps Git repository. I am using TFS-GIT tool. My goal is to keep all TFVC repository changesets in the new Git repository. I followed these steps:
    – I downloaded the TFS-GIT tool lastest binaries.
    – In my DevOps project I created a new Git repository.

    And then I run this commmad:
    git-tfs.exe clone -u {devops-user-email} -p {devops-password} https://{devops-user}.visualstudio.com/{project-name}/_versionControl/$/{tfvc-repository-name}/ https://{devops-user}.visualstudio.com/{project-name}/_git/{new-git-repository-name} . –branches=none

    And I get this output:
    error: Specified git repository directory is not empty

    This is strange because the Git repository is new (no initialization).

    Here are the logs:
    2019-03-01 16:17:32.1986 [Debug] Trying to get HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\DevDiv\vs\Servicing\14.0
    2019-03-01 16:17:32.2202 [Debug] Visual Studio 2015 detected…
    2019-03-01 16:17:32.3828 [Debug] git command: Starting process: git rev-parse –show-prefix
    2019-03-01 16:17:32.3858 [Debug] git command time: [00:00:00.0049889] rev-parse –show-prefix
    2019-03-01 16:17:32.4154 [Debug] Command run:git-tfs.exe clone -u {devops-user-email} -p {devops-password} https://{devops-user}.visualstudio.com/{project-name}/_versionControl/$/{tfvc-repository-name}/ https://{devops-user}.visualstudio.com/{project-name}/_git/{new-git-repository-name} . –branches=none
    2019-03-01 16:17:32.4154 [Debug] No authors file used.
    2019-03-01 16:17:32.4154 [Debug] git-tfs version 0.29.0.0 (TFS client library 14.0.0.0 (MS)) (64-bit)
    2019-03-01 16:17:32.4154 [Debug] GitTfs.Core.GitTfsException: error: Specified git repository directory is not empty
    at GitTfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList1 args)
    at GitTfs.GitTfs.Main(GitTfsCommand command, IList
    1 unparsedArgs)
    at GitTfs.GitTfs.Run(IList`1 args)
    at GitTfs.Program.Main(String[] args)
    2019-03-01 16:17:32.4154 [Error] error: Specified git repository directory is not empty

    Another thing is that I have Visual Studio 2017 intalled and the tool detects Visual Studio 2015.

    Can you help me?

    1. I will try to help you…sure 🙂

      Things you can try or keep in mind:
      – The folder locally on your machine needs to be completely new and empty! Please check that before you start the command in that specific folder.
      – I am not aware of detecting the VS2015 instead of VS2017…If you want to be sure to use the VS2017 TFS assemblies use during teh clone/get of the TFVC files deinstall VS2015 first and try again.
      – If your TFVC repo is allready in Azure DevOps you can use the default import method in Azure DevOps when you create a new Git repo. There are some limitations on this but maybe this is the easyest way to go for you.

      Good luck, and if you any questions or comments, please feel free ta ask again.

      1. Hello Fabian,

        Thank you for reply. My goal is to create this repository on Azure DevOps, instead of on my local machine. I created a new repository in the same project, but I didn’t initialize it, so I think it has to be empty. I don’t have VS 2015 installed. Before to try this I use the tool you mentioned, but I want to save all changesets in the new Git repositoy.

        I thought I had something wrong in my command. I followed this help “Fetch all the history, for just the main branch”: https://github.com/git-tfs/git-tfs/blob/master/doc/usecases/migrate_tfs_to_git.md#fetch-all-the-history-for-just-the-main-branch-ignoring-all-the-other-branches

        It seem that it is no accessing the right Git folder, but I don’t know why.

        1. Now I am trying to do the same on my local machine. I created a new folder (“C:\DATA\Git”) for testing. I am using this command in Windows command prompt:

          git-tfs.exe clone -u {devops-old-account-email} -p {devops-old-account-password} https://{devops-old-account}.visualstudio.com/{project-name}/ $C/DATA/Git . –branches=none

          I get the same “Specified git repository directory is not empty” error, but it is a new empty folder. In logs file I can read this:

          git stderr: fatal: not a git repository (or any of the parent directories): .git

          So I think that I have to initialize the repository on Git. If I run “git init” on git command line, .git folder is created. And running again the same git-tfs command I get the same error, so I think that this $C/DATA/Git is wrong. How can I write the path properly?

          1. You don’t need to specify the local folder! Create a new empty git folder somewhere. The open a command prompt in that folder. Is the the command to execute you only use 2 parameters. The first parameter is the URL to your TFS or Azure DevOp COLLECTION, and the second parameter is the complete path in your EXISTING TFVC repository. Thats why this second parameter starts with “$/”.

            git tfs clone “http://teamserverURL:8080/tfs/COLLECTIONNAME” “$/COMPLETE_PATH_TO_SOURCEFOLDER”

            The local empty folder will be git initialized automatically. Hope this helps.

  2. Ok thank you for your help, I understand better the command parameters. So I changed the command to:

    git tfs clone -u {devops-user-email} -p {devops-password} https://{devops-username}.visualstudio.com/{collection-name} $/{source-folder} . –branches=none

    I use -u and -p because I think that I need to login on Azure DevOps. Now I have another output:

    TF31002: Unable to connect to this Team Foundation Server: https://{devops-username}.visualstudio.com/{collection-name}.
    Team Foundation Server Url: https://{devops-username}.visualstudio.com/{collection-name}.

    Possible reasons for failure include:
    – The name, port number, or protocol for the Team Foundation Server is incorrect.
    – The Team Foundation Server is offline.
    – The password has expired or is incorrect.

    Technical information (for administrator):
    The remote server returned an error: (404) Not Found.

  3. It seems, and I don’t know why, {collection-name} is “DefaultCollection” instead of the name that I see on Azure DevOps. Now I am getting all changesets. Thank you for you help and patience.

Leave a Reply to Jon Cancel reply