Bug #2604

Convert build scripts to use subversion instead of cvs for fetching FreeBSD sources

Added by Jim P 9 months ago. Updated 4 months ago.

Status:New Start date:08/23/2012
Priority:Normal Due date:
Assignee:- % Done:

0%

Category:-
Target version:pfSense - 2.2
Affected version: Affected Architecture:

Description

FreeBSD has ditched cvs and though at the moment there are servers converting svn to cvs, there will be problems fetching certain FreeBSD branches from cvs, and it may not exist entirely for 10.x.[1] Even if cvs were not going away, svn is the path forward.

Due to this, the parts of our build scripts that currently use csup will need to be switched to use svn instead.
The builder documentation will also be need to be updated to reflect subversion as a dependency.

A local svn mirror would be useful, there are some docs on setting one up in the same thread as [1].

1. http://lists.freebsd.org/pipermail/freebsd-stable/2012-August/069236.html

History

#1 Updated by Jim P 6 months ago

FreeBSD announced a security issue here:
http://www.freebsd.org/news/2012-compromise.html

And in that, they state:
  • If you use the already-deprecated cvsup/csup distribution mechanisms, you should stop now.
  • If you were using cvsup/csup for ports, you should switch to portsnap(8) right away. ports developers should be using Subversion already. Further information on preferred mechanisms for obtaining and updating the ports tree can be found at http://www.freebsd.org/doc/handbook/ports-using.html

So this will need to be addressed fairly soon.

#2 Updated by Mathieu Simon 6 months ago

Just as input, since we already use git, and FreeBSD has a well synced mirror on github.com/freebsd which includes
all we need (src, and freebsd-ports). Any interest in there?

I'd see the advantage of having the ability of quicker syncs (with the disadvantage that git initially takes longer to clone a repository though). This way we could 'git checkout -b <cleanbranch> origin/releng/8.3' at every build and have a clean branch for every build (which is what git is quick at).

But I'd see the point in relying in the official channels for getting the FreeBSD sources...

#3 Updated by Mathieu Simon 6 months ago

Actually it seems github could even be considered a relatively official mirror according to: http://wiki.freebsd.org/Git

#4 Updated by Jim P 6 months ago

Ermal and I had talked about that briefly over the weekend. That may end up being better.

The only downside is that it also uses another bridge/connector that could break or tear down rather than using SVN directly.

The main downsides of SVN that I could see so far is speed and the ease with which a repository can be cleanly reset back to the upstream copy. csup did this automatically, with git you can do "git reset --hard origin/branch" but even svn revert and such can still leave bits and pieces around.

#5 Updated by Jim P 6 months ago

Either way, at ~1.5GB per copy checked out, we'd be better off making sure we don't need to blow away the repo and re-download it every time.

#6 Updated by Ermal Luçi 6 months ago

Normally svn supports sparse checkouts.
It also has a mode that can reset the status of the repo to a certain rXXXXX, just like csup.

So that can be mitigated somehow.

#7 Updated by Mathieu Simon 6 months ago

I think most importantly those using it the most should be comfortable with it.

SVN does 'svn export <rev>' quite well too if you want a clean export, git can do 'git archive' in similar way from a repository.

Once decided which route to go, let us know, there are a couple of scripts that will require to be touched and checked... :-)

#8 Updated by Jim P 4 months ago

Ports need updated sooner rather than later. cvs support for the ports tree is being shut down on Feb 28th.

We don't have many places that do use csup on the ports-supfile, but there are a couple. They just need to be switched to portsnap or svn. Portsnap is probably easier.

Also available in: Atom PDF