Jenkins console script with power

Problem

Many times I have been on a jenkins script console wanting to do a shell script using the pipe character to chain calls together.

du -sh --max-depth=1 | sort -h

Jenkins Script console default scripting

The suggested way of doing things leaves much to be desired.  It does give you groovy to do queries in but it is not very powerful.

println "uname -a".execute().text

The Good

Works for default simple single commands.

The Bad

You cannot pipe a command to another and white space in directory names does not work.

Step better

println(['ls', '/tmp/folder with spaces'].execute().text)

The Good

This to allow folders with spaces by turning command into array of strings. You do not have to escape any spaces and it will be able to deal with it.

The Bad

You cannot chain commands together

The better Jenkins script console command

Source of Jenkins shell with pipe discussion

println new ProcessBuilder( 'sh', '-c', 'du -h --max-depth=1 /var/foo/bar/ | sort -hr').redirectErrorStream(true).start().text

The good

Can chain commands together

The Bad

Have to escape whitespace in the java way with \\

println new ProcessBuilder( 'sh', '-c', 'du -h --max-depth=1 /var/foo/bar/folder\\ with\\ spaces | sort -hr').redirectErrorStream(true).start().text
Advertisement

Scala Spray Can using Gradle with Cloud foundry and the java buildpack

What we are trying to do

Use gradle to build and package our app. We are using gradle as our build tool as we did not like the Sbt way of doing things.

Deploy a single self contained running Spray application with all required runtimes dependency on Cloud Foundry as a java application.

If possible get off using our self written wrapper java tools to deploy and manipulate Cloud Foundry using the cloud foundry jar.

Background

We were trying out different gradle plugins to try and get our Spray app deployed.  We had settled on oneJar as there was documentation of people using it with gradle to create and run a self contained file.

Onejar

The oneJar plugin changes the class loader to accept self contained single file that is a jar with jars inside it.

Messing around with class loading means you are going to have a bad time.

An idea from Spring Boot deployment

Chatting with some other devs I was also explaining we had problems deploying our Spring boot app, so were prevented from deploying onto our latest and greatest Cloud Foundry instance.  We have hard gained experience on out how to deploy our apps on an older version of CF but it never started properly on the new CF.  I was pointed to this by another developer as to how to deploy a Spring Boot application properly from the java buildpack documentation.

Cloud Foundry docs for Spring boot deployment

DistZip to the Rescue

This got me thinking of how Spring boot uses the distZip style deployment and could I leverage it to deploy my Spray app. Spring boot uses the distZip which comes from the  application gradle plugin.

What distZip does

Creates a zip file with all the runtimes needed your application with the correct  META-INF/MANIFEST.MF info and a start script to run it.  Cloud Foundry is happy to deploy this.

You will need to set your main class to tell distZip what to run

mainClassName = "org.gradle.sample.Main"

Success

On Cloud Foundry the application needs to bind to a port provided on deployment time to accept connections, else CF will not let it succeed in starting and kill it.

Here is a code snippet of what it takes to bind the Spray to CF.


import com.typesafe.config._
....

val envConf: Config = ConfigFactory.systemEnvironment()
def serverPort = envConf.getInt("VCAP_APP_PORT")

....
 IO(Http) ? Http.Bind(actorSystem.actorOf(Props(wire[YourActor]), "your-actor"), interface = "0.0.0.0", port = serverPort)
}

Using the Cloud Foundry Gradle Plugin

After getting a successful deployment using the command line tool(CLI) I was then able to use the Gradle CF plugin to configure and deploy.  This makes it easier for Jenkins as we do not have to change our build slaves to deploy the CLI and change our chef scripts for build slaves.

Docs for CF gradle plugin

Setup snippet

<pre>
cloudfoundry {
    target = "https://YOUR_INSTANCE.com"
    space = "snapshot-YOURS"
    file = file("./build/distributions/YOUR_DIST_ZIP.zip")
    buildpack = "https://github.com/cloudfoundry/java-buildpack.git#v3.2"
    application = "YOUR_APP_NAME-dev"
    trustSelfSignedCerts = true // needed for our managed instance
    memory = 1024
    instances = 2
    env = ["SPRING_PROFILES_ACTIVE": "A_PROFILE"]  //we are using the spring boot style way of dealing with different properties files for different enviornments.
}</pre>

CF Commands

It is a bad idea to hold the credentials in your source control system. so you or your build slave  should login and out using secured credentials.

  1. gradle cfLogin -PcfUsername=’your-user-name’ -PcfPassword=’your-password’  -PcfOrganization=’your-organisation’

With the above snippet the target, space, buildpack, file and application name you can just do a push without any additional commands.

2. gradle cfPush

Different Environments

Just override the needed elements to deploy to new environments with -PcfThingToOverride=’NEW_VALUE’

E.G.:

gradle cfPush -PcfTarget=’https://staging.your.company.com&#8217; -PcfSpace=’stage-space’ -PcfInstances=’4′ -PcfApplication=’YOUR_APP-stage’ ….

 

 

 

Unit testing $scope.$on(‘$viewContentLoaded’) on an Angularjs Controller

TL;DR = call

$scope.$broadcast('$viewContentLoaded');

in your unit test. This will call your

$scope.$on('$viewContentLoaded', function () {}); 

method in your controller

If you have a Controller that has a method that is not fired until the view is loaded you will have a method similar to the following:

    $scope.$on('$viewContentLoaded', function () {
       SomeService.callMethod(params);
    });

The test case needs to wire in the $scope into your test like below

describe('when the campaign controller is initialized', function () {

 beforeEach( function() {
    $scope.$broadcast('$viewContentLoaded');
 } );

 it('should call the getAllCampaignTypes service call', function () {
     expect(mockedSomeService.callMethods).toHaveBeenCalledWith( ... );
 });

The Most important thing is the call to $scope.$broadcast(‘$viewContentLoaded’);

Yeoman grunt-ngmin to grunt-ng-annotate save massive time on your grunt build.

Background

If you have build your skeleton app from yeoman using the angularjs-generator and you created it before August 3rd 2014  you are running ngmin task.

Ngmin has been deprecated and you are told this every time you do an npm install on the project.

Git Issue that resolves this for new generated projects from yeoman

https://github.com/yeoman/generator-angular/issues/801

Why to migrate.

ng-min is deprecated so it is no longer supported.

ng-min is slow.  For a mature medium-sized angularjs app ng-min was taking 50 seconds of the build time.  Running with ng-annotate instead it now takes only 2 seconds to do the same thing.

Now the uglify task takes the most time of the build at 8 seconds against the 2 seconds needed for the ngannotate task.

A savings of 48 seconds a build.  

Built Machine spec:

OSX 10.9.5 with 3.2 GHz with 24 GB of ram.  No modifications have been made to the memory settings for grunt or node.

Continue reading

Jenkins Fixing SCM Sync with Git and Failing with Delete Jobs

https://wiki.jenkins-ci.org/display/JENKINS/SCM+Sync+configuration+plugin

I have encountered this several times where if you try and change a job name along with any configuration for a job Git will get into a bad state and the SCM will start throwing up errors like:

delete Jobs/OLD_JOB_NAME 

The only thing I have found is to fix this is to follow the following steps:

You will have to have admin rights and the ability to restart your jenkins.

  1. recreate the OLD_JOB_NAME
  2. change the system configuration scm sync to none.
  3. restart the jenkins.  via your_jenkins_server/restart
    1. change the system configuration scm sync to git.
  4. Ensure that the configuration is now syncing correctly  and you do not have any more errors.
  5. delete OLD_JOB_NAME again.
  6. If this still does not work you can try downgrading the plugin and repeating the steps again.

Warning:

You may lose all your job configurations after you reenable the SCM sync so you will have to make a small change to each job name such as changing the description by a single letter.

 

 

Saving csv files from a Nodejs project

Update: package available on bower.  bower -S FileSaver.

I am requesting a csv from another service and I wanted to be able to prompt the customer to save the csv for use later.  I managed to get it to save the csv but the filename could not be set so it was not useful.  I found the following library to help out with this.

https://github.com/eligrey/FileSaver.js

 

Angularjs Custom Form Validation

update: 9/9/14 added the ng-learn link which has  a flushed out example. If you are doing this you will need to change you test cases and include all the things like you were testing a directive.

If you want to set an input value to be invalid before submission then this is a good resource.

$scope.yourForm.nameOfElement.$setValidity( ‘reason’,  false );

E.G.: You want to add an error to your start date in a search form.

$scope.searchForm.startDate.$setValidity( ‘date’, false );

http://aleaiactaest.ch/2012/08/23/custom-form-validation-in-angular-js/

To see a more fully explained example please see:

http://ng-learn.org/2014/03/Writing_Custom_Validations_Part4/

Click the launch demo button.  Then click the script.js and the spec.js to see the full example.

Managed Script plugin and SCM sync on Jenkins

If you are using https://wiki.jenkins-ci.org/display/JENKINS/Managed+Script+Plugin  and the https://wiki.jenkins-ci.org/display/JENKINS/SCM+Sync+configuration+plugin

You will have to add a Manual synchronization includes: 

buildstep-config-files.xml

to get it to save the managed files configuration.  Otherwise all your managed script files you share across jobs will be lost if you lose the jenkins server and have to rebuild it.

 

IE8 Angularjs

I had a problem with IE8 compatibility mode going to document Mode: IE 7 standards. This caused my javascript to act funny and radio buttons not to work.
I was also getting the error pop up too:

https://docs.angularjs.org/api/ng/service/$sce

I found the following on stack overflow that is the key as my project is using Html5.
I have the in my project

http://stackoverflow.com/a/7685060/708860

Excerpt of what I did that worked.

Note the the [if] tag before the <!doctype html>

<!--[if HTML5]><![endif]-->
<!doctype html>
<!--[if the boilerplate conditionals goes here<![endif]-->
<head>
<!--[if !HTML5]>
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<![endif]-->