Entries tagged [subversion]
The ASF CMS
Over the past 3 months, the Infrastructure Team has developed and deployed a custom CMS for Apache projects to use to manage their websites. There is a document available which explains the rationale, role, and future plans for the CMS. We have opened up the ACLs for the www.apache.org site for all committers to now be able to edit content on the site using the cms (while still restricting live publication to the Apache membership and the Infrastructure Team).
Your commit will trigger buildbot to build a staging version of your changes. You can follow the build while it is ongoing, and once it has completed you can click on the [Staged] link to see the results. Members and Infrastructure Team members can continue on and publish those changes once they are satisfied with them. Other committers may need to send a note to the site-dev@ mailing list to request publication of their changes.
The publication links in the CMS are essentially merge + commit operations in subversion which are tied into the live site via svnpubsub. That means publishing in the CMS is virtually instantaneous.
The CMS is now open to all top-level and incubating projects. Interested projects should contact the infrastructure@ mailing list or simply file an INFRA ticket against the CMS component. Early adopters are encouraged to collaborate on the wiki page for working out usage and adoption issues.
Posted at 04:25AM Dec 02, 2010 by joes in General | |
1 million commits and still going strong.
Yesterday, the main ASF SVN code repository passed the 1 million commit mark. Shortly thereafter one of the ASF members enquired as to how he could best grab the svn log entries for all of these commits. As always there were a bunch of useful replies, but they were all set to take quite some time; mainly because if anyone just simply runs
svn log http://svn.apache.org/repos/asf -r1:1000000
It will not only take several hours, it will also cause high levels of load on one of the two geo-balanced SVN servers. Also, requesting that many log entries will likely result in your IP address being banned.
So I decided to create the log set locally on one of the SVN servers. This is now available for download [http://s.apache.org/1m-svnlog] [md5]
This is a 50Mb tar/gz file. It will uncompress to about 240Mb. The log 'only' contains the log entries from 1 -> 1,000,000 - if you want the rest you can run:
svn log http://svn.apache.org/repos/asf -r1000001:HEAD
This will give you all the log entries from 1M+1 to current
Posted at 11:55AM Sep 23, 2010 by pctony in Status | |
LDAP, groups and SVN - Coupled together
The infrastructure team have now completed the next stage of the planned LDAP migration.
We have migrated our old SVN authorisation file, and POSIX groups into LDAP data. SVN access control is now managed using these groups.
This means to change access the Subversion repositories is now as simple as changing group membership. We use some custom perl scripts that build the equivalent authorisation file meaning that we dont need to use the <location> blocks nasty hack to do this. It also means that all changes, including adding new groups and extending access control is made simple.
ASF PMC chairs, are now able to make changes to their POSIX, and SVN groups whilst logged into people.apache.org - using a selection of scripts:
All of these scripts have a '--help' option to show you how to use them.
What's next? We are now working on adding a custom ASF LDAP schema, that will allow us to record ASF specific data such as ICLA files and date of membership etc.
We will also be looking at adding support for 3rd party applications such as Hudson, and building an identity management portal where people can manage their own account.
Posted at 10:03PM Feb 22, 2010 by pctony in Status | |
Improving our Subversion Services
We are using pgeodns, the same software that powers CPAN Search and the NTP Pool. With pgeodns we can give out different DNS entries to clients, depending on where they are connecting from. It isn't an exact science, but for most clients it is good enough to find the closer Subversion Server.
Users in North America are directed to Eris, our traditional Subversion Master Server. Eris is a Dell 2950 also running FreeBSD 7.0, using ZFS raid2z over 4 disks, hosted in Corvallis, Oregon at OSUOSL.
Read operations are handled on the nearest mirror, and are much faster for everything from the initial checking out to running an update due to the decreased latency.
While this change should improve the experience significantly, we have some other changes coming up soon for svn.apache.org:
- Upgrade to Subversion 1.6: Representation Sharing, inode packing, and memcached support should help make our SVN servers even faster.
- Upgrade both Eris and Harmonia to FreeBSD 7.2-STABLE: The ZFS filesystem is experimental in FreeBSD 7, and there are many stability and performance enhancements available in newer versions.
- Adding more Geographic Mirrors: Once we are comfortable with the current setup, we would like to expand to another mirror location, hopefully in Australia or Asia.
Posted at 08:47PM Apr 02, 2009 by pquerna in General | |