Entries tagged [github]

Tuesday April 30, 2019

Apache and GitHub - a friendly PSA about awesomeness

With the news of the Apache Software Foundation teaming up more closely with GitHub, we feel it natural to elaborate a bit on what has been going on and what this means for you as a committer and/or user of Apache software.

A little bit of history

The Apache Software Foundation started experimenting with git as a source code repository system in 2008, and ventured into GitHub in 2010, where we were graciously offered whatever resources we needed.

At first, this was merely a mirror of our existing git and subversion repositories, but as time went on, and projects expressed an interest in utilizing the many user-friendly features of GitHub, we started work on enabling projects to make proper use of GitHub some three years ago in the middle of 2016. This project, aptly named `gitbox`, ensured that committers could make full use of the GitHub features, while we kept a place within our own infrastructure for people inclined to continue using our infrastructure for their work. As git is decentralized by its very nature, we were able to use GitHub to augment rather than replace our git workflow, bringing our software development to the millions of users on GitHub in addition to the existing Apache community and committers, on a case-by-case basis.

In 2018, we made the decision to combine the two different git service offerings we had into one service, allowing all Apache projects to use GitHub if they so desired. Before then, we had two distinct git services: gitbox and git-wip-us, the initial git service that had been available since 2010. We coordinated the move from git-wip to gitbox with the various Apache projects, and in early 2019 we had migrated all projects to the new service, enabling GitHub features for all git-based Apache projects.

With Microsoft's acquisition of GitHub in 2018, and their commitment to help strengthen open source development, we have received additional resources to help lower the bar for contributions, and we'd like to thank GitHub for their support of the Apache Software Foundation through all nine years of using their platform.

What this means for you as a committer

As stated above, our GitHub integration is an augmentation of our existing service. It is available to all committers on git-based projects to make use of, should they so wish. All new git repositories will automatically be available on both GitHub and Gitbox.

For those wishing to take full advantage of GitHub's features, one can link their GitHub and Apache accounts through https://gitbox.apache.org/setup/ which will grant their GitHub account write access to the repositories you'd traditionally have access to at Apache.

People that wish to continue using their Apache committer accounts to commit code may continue doing so on gitbox.apache.org with their Apache credentials. Nothing has changed in that respect.

As Apache is a very email-centered organization, all GitHub activity is naturally linked to our mailing lists to ensure the same level of openness in the development of our software.

What this means for you as a user of Apache software

For many projects, the move to GitHub means a lower bar to both contributing as well as troubleshooting and submitting issues to the projects, through the GitHub issue and pull request features.

Our commitment to provenance, quality and open governance remains the same, and with our tight integration with GitHub through our linked account service, we are able to bring what made Apache a mark of quality to the many users and contributors on GitHub.

As always, if you have any questions, comments, remarks or feedback about this, we welcome you to reach out to the Apache Infrastructure Team at: users@infra.apache.org

Sunday March 26, 2017

Bringing GitPubSub to the Apache Jenkins build server

When it comes to [Jenkins], it has long been known that [polling must die].

While we could go and create post commit hooks in all the ASF hosted Git repositories, that is something that realistically is just creating an added maintenance burden. In any case, we have [GitPubSub].

The question then becomes, how do we integrate [GitPubSub] with [Jenkins]? Thankfully, ASF committer stephenc is also an active committer to the [Jenkins] project and created a [plugin] that connects to [GitPubSub] parses the events and passes them through to the Jenkins [SCM API].

What does this mean?

* You can turn your Git polling down - way way down - to something like once per day. This should significantly reduce the load on both the ASF git servers and builds.apache.org
* Your builds will be triggered in seconds rather than having to wait for the next polling run.
* You can try out using Multi-branch projects much like the [Maven] project has been doing for [Maven core] and [Maven Surefire]

If the reaction to this change proves positive, the next step will be to integrate SvnPubSub with Jenkins and bring the benefits to the Subversion based projects too

See also this blog post by Stephen Connolly:


[polling must die]: http://kohsuke.org/2011/12/01/polling-must-die-triggering-jenkins-builds-from-a-git-hook/
[GitPubSub]: https://www.apache.org/dev/gitpubsub.html
[Jenkins]: https://jenkins.io/
[plugin]: https://github.com/stephenc/asf-gitpubsub-jenkins-plugin
[SCM API]: https://plugins.jenkins.io/scm-api
[Maven]: https://maven.apache.org
[Maven core]: https://builds.apache.org/job/maven-3.x-jenkinsfile/
[Maven Surefire]: https://builds.apache.org/job/maven-surefire-pipeline/

Posted on behalf of Committer Stephen Connolly (stephenc)

Thursday October 02, 2014

GitHub pull request builds now available on builds.apache.org

The ASF Infrastructure team is happy to announce that you can now set up jobs on builds.apache.org to listen for pull requests to github.com/apache repositories, build that pull request’s changes, and then comment on the pull request with the build’s results. This is done using the Jenkins Enterprise GitHub pull request builder plugin, generously provided to the ASF by our friends at CloudBees. We've set up the necessary hooks on all github.com/apache repositories that are up as of Wednesday, Oct 1, 2014, and will be adding the hooks to all new repositories going forward.

Here’s what you need to do to set it up:

  • Create a new job, probably copied from an existing job.
  • Make sure you’re not doing any “mvn deploy” or equivalent in the new job - this job shouldn’t be deploying any artifacts to Nexus, etc.
  • Check the "Enable Git validated merge support” box - you can leave the first few fields set to their default, since we’re not actually pushing anything. This is just required to get the pull request builder to register correctly.
  • Set the “GitHub project” field to the HTTP URL for your repository - i.e.,"http://github.com/apache/incubator-brooklyn/"- make sure it ends with that trailing slash and doesn’t include .git, etc.
  • In the Git SCM section of the job configuration, set the repository URL to point to the GitHub git:// URL for your repository - i.e., git://github.com/apache/incubator-brooklyn.git.
  • You should be able to leave the “Branches to build” field as is - this won’t be relevant anyway.
  • Click the “Add” button in “Additional Behaviors” and choose "Strategy for choosing what to build”. Make sure the choosing strategy is set to “Build commits submitted for validated merge”.
  • Uncheck any existing build triggers - this shouldn’t be running on a schedule, polling, running when SNAPSHOT dependencies are built, etc.
  • Check the “Build pull requests to the repository” option in the build triggers.
  • Optionally change anything else in the job that you’d like to be different for a pull request build than for a normal build - i.e., any downstream build triggers should probably be removed,  you may want to change email recipients, etc.
  • Save, and you’re done!

Now when a pull request is opened or new changes are pushed to an existing pull request to your repository, this job will be triggered, and it will build the pull request. A link will be added to the pull request in the list of builds for the job, and when the build completes, Jenkins will comment on the pull request with the build result and a link to the build at builds.apache.org

In addition, you can also use the "Build when a change is pushed to GitHub" option in the build triggers for non-pull request jobs, instead of polling - Jenkins receives notifications from GitHub whenever one of our repositories has been pushed to. Jenkins can then determine which jobs use that repository and the branch that was pushed to, and trigger the appropriate build.

If you have any questions or problems, please email builds@apache.org or open a BUILDS JIRA at issues.apache.org

Wednesday February 12, 2014

Improved integration between Apache and GitHub

After a few weeks of hard work and mind-boggling debugging, we are pleased to announce tighter and smarter integration between GitHub and the Apache Software Foundation's infrastructure.

These new features mean a much higher level of replication and retention of what goes on on GitHub, which in turns both help projects maintain control over what goes on within their project, as well as keeping a record of everything that's happening in the development of a project, whether it be on ASF hardware or off-site on GitHub.

To be more precise, these new features allows for the following:

  • Any Pull Request that gets opened, closed, reopened or commented on now gets recorded on the project's mailing list
  • If a project has a JIRA instance, any PRs or comments on PRs that include a JIRA ticket ID will trigger an update on that specific ticket
  • Replying to a GitHub comment on the dev@ mailing list will trigger a comment being placed on GitHub (yes, it works both ways!)
  • GitHub activity can now be relayed to IRC channels on the Freenode network.

As with most of our things, this is an opt-in feature. If you are in a project that would like to take advantage of these new features, please contact infrastructure, preferably by filing a JIRA ticket with the component set to Git, and specifying which of the new features you would like to see enabled for your project.

On behalf of the Infrastructure Team, I hope you will find these new features useful and be mindful in your use of them.



Hot Blogs (today's hits)

Tag Cloud