Apache Infrastructure Team
Bringing GitPubSub to the Apache Jenkins build server
When it comes to [Jenkins], it has long been known that [polling must die].
While we could go and create post commit hooks in all the ASF hosted Git repositories, that is something that realistically is just creating an added maintenance burden. In any case, we have [GitPubSub].
The question then becomes, how do we integrate [GitPubSub] with [Jenkins]? Thankfully, ASF committer stephenc is also an active committer to the [Jenkins] project and created a [plugin] that connects to [GitPubSub] parses the events and passes them through to the Jenkins [SCM API].
What does this mean?
* You can turn your Git polling down - way way down - to something like once per day.
This should significantly reduce the load on both the ASF git servers and builds.apache.org
* Your builds will be triggered in seconds rather than having to wait for the next polling run.
* You can try out using Multi-branch projects much like the [Maven] project has been doing for [Maven core] and [Maven Surefire]
If the reaction to this change proves positive, the next step will be to integrate SvnPubSub with Jenkins and bring the benefits to the Subversion based projects too
See also this blog post by Stephen Connolly:
[polling must die]: http://kohsuke.org/2011/12/01/polling-must-die-triggering-jenkins-builds-from-a-git-hook/
[SCM API]: https://plugins.jenkins.io/scm-api
[Maven core]: https://builds.apache.org/job/maven-3.x-jenkinsfile/
[Maven Surefire]: https://builds.apache.org/job/maven-surefire-pipeline/
Posted on behalf of Committer Stephen Connolly (stephenc)
Posted at 01:07AM Mar 26, 2017 by gmcdonald in Development | |
blogs.a.o moved, upgraded and improved
blogs.apache.org - the site you are reading now! has had a bit of an update.
1. We moved it from an aged VM Host to the Cloud (thanks LeaseWeb!)
2. We puppetised the entire service, from install to deploy (see our Github Mirror )
3. We upgraded the Apache Roller software from 5.0.3 to the latest 5.1.2
4. We enabled LDAP for logins. That's right! Every single ASF Committer can now just login! No more creating an INFRA Jira ticket just to get a Roller account on blogs.apache.org
Other stuff remains the same - meaning if you are a Blog Administrator you still need to invite committers into your blog, you still need to choose to make them an Author or Admin etc - Roller doesnt support anything more than login auth for LDAP currently - but I bet the project would love to see the LDAP integration extended and improved if you feel the need!.
Anyhow, our first new year present to our ASF Committers, a shiny updated blog instance,
Enjoy, and have a great 2017!!
Committer shell access to people.apache.org
Apache committers are granted shell access to a host know as either people.apache.org or minotaur. As you may know, there has been a two year grace period in which we have advertised the upcoming change away from password logins to SSH key only.
Due to a recent significant increase in security issues, the Infrastructure team has taken steps to complete the implementation of key-only logins to protect ASF computing resources.
If you can't access the host anymore then it is very likely you do not have your key stored in LDAP. Please check your LDAP data in https://id.apache.org - and add your key(s) if they are not present. If neccessary, ensure your keys are loaded locally (for linux see http://linux.die.net/man/1/ssh-add and http://linux.die.net/man/1/ssh-agent)
The host will pick up this change within 5 minutes of you making your change and you should be able to get in again.
As always if you have any issues please open a JIRA issue in the INFRA project and we will help you as soon as we can.
Posted at 11:38PM Sep 25, 2014 by pctony in Development | |
Committers mail relay service
For a very long time now we have allowed committers to send email from their @apache.org email address from any host. 10 years ago this was less of an issue than it is today. In the current world of mass spam and junk flying around, mail server providers are trying to find better ways to implement a sense of safety from this for their users. One such method is SPF . These methodologies check that incoming email actually originated via a valid mail server for the senders domain.
For example if you send from email@example.com, but you just send that via your ISP at home, it could be construed as being junk as it never came via an apache.org mail server. Some time ago we setup a service on people.apache.org to cater for this, but it was never enforced and it seems that the SMTP daemon running the service is not 100% RFC compliant and thus some people have been unable to use this service.
As of today, we have stood up a new service on host mail-relay.apache.org that will allow committers to send their apache.org emails via a daemon that is RFC compliant and uses your LDAP credentials. You can read here  what settings you will need to be able to use this service.
On Friday October 10th, at 13:00 UTC the old service on people.apache.org will be terminated, and the updates to the DNS to enforce sending of all apache.org email to have originated via an ASF mail server will be enabled. This means that as of this time if you do not send your apache.org email via mail-relay it is very likely that the mail will not reach it's destination.
When we say 'send your apache.org email' - we mean that when you send *from* your firstname.lastname@example.org email. Emails sent *to* any apache.org email address will not affected by this.
Improved integration between Apache and GitHub
After a few weeks of hard work and mind-boggling debugging, we are pleased to announce tighter and smarter integration between GitHub and the Apache Software Foundation's infrastructure.
These new features mean a much higher level of replication and retention of what goes on on GitHub, which in turns both help projects maintain control over what goes on within their project, as well as keeping a record of everything that's happening in the development of a project, whether it be on ASF hardware or off-site on GitHub.
To be more precise, these new features allows for the following:
- Any Pull Request that gets opened, closed, reopened or commented on now gets recorded on the project's mailing list
- If a project has a JIRA instance, any PRs or comments on PRs that include a JIRA ticket ID will trigger an update on that specific ticket
- Replying to a GitHub comment on the dev@ mailing list will trigger a comment being placed on GitHub (yes, it works both ways!)
- GitHub activity can now be relayed to IRC channels on the Freenode network.
As with most of our things, this is an opt-in feature. If you are in a project that would like to take advantage of these new features, please contact infrastructure, preferably by filing a JIRA ticket with the component set to Git, and specifying which of the new features you would like to see enabled for your project.
On behalf of the Infrastructure Team, I hope you will find these new features useful and be mindful in your use of them.
Posted at 01:16AM Feb 12, 2014 by humbedooh in Development | |
https://id.apache.org -- New Password Service
The infrastructure team are pleased to announce the availability of id.apache.org the new password management tool for all ASF committers and members. This new service will allow users to:
- Reset forgotten LDAP passwords themselves, no need to contact the Infra team anymore.
- The ability to change their LDAP password.
- The ability to update your LDAP record, i.e. change forename, surname or mail attributes. .
Once logged in you will note that some fields are not editable, this is by design and are there merely to show you your LDAP entry. You are currently only allowed to edit your Surname, Given name (Forename), and Mail attributes. This list may be extended as we make more features available, and they will be announced as and when.
Users of this service should note that we have a few small bugs to iron out, and this will be done as soon as possible. For example if you attempt to modify your details and do no re-enter your password you will currently see a generic HTTP 500 error.
Thanks must go to Ian Boston (ieb), and Daniel Shahaf (danielsh) for making this work. Ian provided the initial code (his first ever attempt at Python too). Daniel then took it and implemented several changes and generally improved the backend.
 - It should be noted that updating your mail record in LDAP will not currently have any affect on where your apache.org email is forwarded on too. This is planned to take place later this year.
Posted at 04:36PM Jan 14, 2011 by pctony in Development | |
LDAP and password policy
As of approximately 03:00 (UTC) today the infrastructure team have enabled a password policy for all LDAP accounts.
This policy has been implemented at the LDAP infrastructure level and will affect all users. It has been deployed using OpenLDAP's password policy schema, and overlay.
At the time of launch we will be enforcing the following policy.
- At the time of a given users 10th successive login failure the account will be locked.
- The account will then be automatically unlocked 24 hours later, or until a member of root@ unlocks it for you.
- If the user successfully completes a login before the tally reaches 10, the counter for failed logins is reset back to 0.
We are enabling this to try and prevent any brute force attempt at guessing passwords. It will also highlight potential issues with accounts.
As with all account related queries, you should be contacting root@ - We will be able to unlock your account for you, allowing you to gain access.
Posted at 06:38AM Dec 17, 2010 by pctony in Development | |
ASF Buildbot svn setup
Here at the ASF we have a subversion setup with all our projects code in one repository, with each of those projects having their own style of trunk/branches/tags/site etc.. This works well for us, but did present us with some initial problems when setting up our Buildbot instance to work with it.[Read More]
Posted at 10:25AM Mar 29, 2010 by administrator in Development | |