GitFirst for Qlik Sense (Beta) - Docs

By using this software, you agree to be bound by the terms and conditions of the EULA.

Quick Links

Prerequisites
Build Context
Files
- Required
  - build.yaml
  - load-script.qvs
- Optional
Build Command
Github Action Example
Azure Devops Pipeline Example

Prerequisites

Before getting started, please ensure that:

There are not any apps in your tenant where the name begins with ‘Managed_’. (see note below).
You have a licence to test this beta version of GitFirst for Qlik Sense (sign up for newsletter).
You have a tenant API key for your Qlik Sense tenant.

Note also:

Where templating is mentioned below, it refers to golang templating. Use of templating is entirely optional but may open up useful solutions.
Minimal and more sophisticated versions of the code below can be found on this branch and this branch respectively in GitHub.

Build Context

The following ‘build context’ is available when templating in addition to anything additional you specify in a values.yaml file (see below):

buildTrigger: Command_Line
git:
  refName: main
  refType: branch
  shaShort: a8b4bda
  sha: a8b4bda15b74ed34758fbd3cd43a50c9c89612b0

When running a command line build, the default value of buildTrigger can be overidden with the --buildTrigger flag.

Files

Required:

build.yaml
load-script.qvs

Optional:

values.yaml
master-items.yaml
tests.yaml

Required Files

build.yaml

This file configures how the app is created in the tenant and which stages run when ‘building’ the app.

A minimal example is shown below.

IMPORTANT: GitFirst will create this app for you (and possibly delete an earlier version before re-creating depending on your config.) Currently it is required that any apps managed by GitFirst have the prefix ‘Managed_’.

build.yaml

containerApp:
  appName: Managed_GitFirst_TestApp

The more advanced example below shows how the build context can be used via some templating code. It also turns on the test stage (which would require additionally a tests.yaml file) and shows commented out various other options.

build.yaml

containerApp:
  # By default a new app with this name is created in the Qlik tenant and is replaced each
  # time a build is made. Further options below can be uncommented to override this behaviour.
  #
  # Note that you have access to the build context and values YAML inside this file - here we 
  # make the git refname part of the file name.
  appName: Managed_GitFirst_TestApp_{{.git.refName}}
  # replaceExisting: true
  # deleteOnFinish: false
  # Can also uncomment one of the following as a QVF app to copy.
  # copyExistingApp:
  #   appName: app_name_to_copy_in_tenant
  # importAppInRepo:
  #   path: you_app.qvf
  # importFromHttp:
  #   url: https://url-to-your-app/app.qvf
  # # Coming soon - import from GitHub release asset.

stages:
  # checkScript:
  #   run: true
  reload:
    run: true # true | false | onlyIfAppNew
    #timeLimit: 15m
  setMasterItems:
    run: true
  runTests:
    run: true
    #timeLimit: 15m

load-script.qvs

The other required file is a load script. You can use a Visual Code extension such as this one to provide some syntax highlighting and intellisense for your Qlik scripts.

load-script.qvs

trace Build triggered by '{{ .buildTrigger }}';
trace shaShort={{ .git.shaShort }}, sha={{ .git.sha }}, refName='{{ .git.refName }}', refType='{{ .git.refType }}';

Cameras:
LOAD * inline [
Customer Product UnitSales UnitPrice
Imagine Film 4 16
Imagine Film 10 15
Imagine Shutter 9 9
PhotoInc Shutter 5 10
PhotoInc Lens 2 20
PhotoInc Magnifier 4 25
Gallery Film 8 15
Gallery Lens 7 19
] (delimiter is ' ');

You have access to templating in this file with the build context and any additional values YAML you provide.

Optional Files

master-items.yaml

Use this file to set master dimensions and measures and variables in the app. An example is shown below. Make sure you also enable this stage in build.yaml.

master-items.yaml

# dimensions:
#   - id: dim1
#     label: Your Dimension
#     expression: = ...your dimension...
measures:
  - id: avgPrice
    label: Average Price
    expression: avg(UnitPrice)
  - id: maxPrice
    label: Max Price
    expression: max(UnitPrice)
variables:
  - id: buildSha
    label: Build SHA
    expression: '{{.git.sha}}'

tests.yaml

Use this file to run tests on your output. Make sure you also enable this stage in build.yaml.

Tests are one of the most powerful features of the product, allowing quick automated regression testing to ensure your final Qlik data model and other aspects of your Qlik output and environment remain stable and bug free. Let us know what sort of tests and checks you would like to see added.

Notes:

In the expressionChecks section, you should be able to run any valid Qlik expression. Note in some cases " quotes are needed to avoid values being interpreted as YAML arrays.
We include here a simple double check that the commit SHA used for the build matches that set earlier as a variable within the app.
The displayOnly is useful to get a quick output in the logs of the actual expression value if you are having trouble creating the tests.
fieldsShouldExist and fieldsShouldNotExist illustrate different ways collections/arrays can be represented in YAML.

tests.yaml

expressionChecks:
  shouldBeTrue:
    - count(DISTINCT $Table) = 1
    - count(DISTINCT $Field) = 4
    - "[Average Price] > 10"
    - "[Max Price] > [Average Price]"
    - "[Build SHA] = '{{.git.sha}}'"
  displayOnly:
    - count(DISTINCT $Table)
    - count(DISTINCT $Field)
    - "[Average Price]"
    - "[Max Price]"
    - "[Build SHA]"
modelChecks:
  allowSyntheticKeys: false
  tablesShouldExist:
    - Cameras
  tablesShouldNotExist:
    - MadeUpTable
  fieldsShouldExist: [Customer, Product, UnitSales, UnitPrice]
  fieldsShouldNotExist:
    - MadeUpField1
    - MadeUpField2
  # For possible tags, see:
  # https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_Hub/Scripting/field-tags.htm
  fieldsShouldHaveTags:
    - fields: [Customer]
      tags: [$text]
    - fields: [UnitSales, UnitPrice]
      tags: [$integer]

values.yaml

You are free to create an arbitrary YAML file named values.yaml.

In this file you will also have access to templating using the build context discussed above. The result of templating out this file will then be available to templating in the load-script.qvs, master-items and tests YAML files.

As an example - we could create this:

values.yaml

maxRows: 5
storeQvds: true
qvdFile: MyQVDs_{{.git.refName}}.qvd

The following data will now be available in our other files to use in templating:

maxRows: 5
storeQvds: true
qvdFile: MyQVDs_main.qvd
buildTrigger: Command_Line
git:
  refName: main
  refType: branch
  shaShort: a8b4bda
  sha: a8b4bda15b74ed34758fbd3cd43a50c9c89612b0

We could now update the load script to use this:

load-script.qvs

Cameras:
FIRST {{.maxRows}} LOAD * inline [
Customer Product UnitSales UnitPrice
Imagine Film 4 16
Imagine Film 10 15
Imagine Shutter 9 9
PhotoInc Shutter 5 10
PhotoInc Lens 2 20
PhotoInc Magnifier 4 25
Gallery Film 8 15
Gallery Lens 7 19
] (delimiter is ' ');

{{if .storeQvds}}
store Cameras into [lib://DataFiles/{{.qvdFile}}];
{{end}}

Build Command

With at least the minimum required files above in place, you should be able to run the following command.

gitfirst run build --acceptEula false --tenantDomain YOUR_TENANT.XX.qlikcloud.com --refType YOUR_REF_TYPE --refName YOUR_REF --sha YOUR_SHA --path YOUR_PATH

Note:

You will first need to ensure that you have LICENCE (see prerequisites) and TENANT_API_KEY (see here) environment variables set.
You will need to change the above --acceptEula flag to true if you accept the EULA.
The YOUR_... values would typically be provided from your build server (e.g. Github Actions, Azure Devops, CircleCI, Jenkins etc.)
Add the --saveOutputs flag to save output files to a directory named gitfirst-outputs.
Add --buildTrigger YOUR_TRIGGER_NAME to override the buildTrigger value in the build context (see above).

Github Action Example

The following is an example Github Action for running a build which will allow you to deploy an app directly from GitHub into your Qlik Sense Saas cloud tenant.

Note:

You will need to update with the correct download URLs and checksums.
You will need to change the --acceptEula flag to true if you accept the EULA.
You will need to ensure that you have LICENCE (see prerequisites) and TENANT_API_KEY (see here) environment variables set (as Github Action secrets).
The optional --saveOutputs flag is included as well as an additional archive-outputs stage to archive these.
The optional --buildTrigger is also included providing a custom value then available to templating code.

.github/workflows/main.yml

on: [push]
jobs:
  build-linux:
    runs-on: ubuntu-18.04
    env:
      TENANT_API_KEY: ${{secrets.TENANT_API_KEY}}
      LICENCE: ${{secrets.LICENCE}}
      TENANT: 'YOUR_TENANT.XX.qlikcloud.com'
    steps:
      - name: checkout
        uses: actions/checkout@v3
      - name: download-tooling
        shell: sh
        run: |
          curl --fail https://...download url.../gitfirst > gitfirst
          sha256sum ./gitfirst
          echo "...checksum from download page..." | sha256sum --check
          chmod +x ./gitfirst          
      - name: build-qlik-app
        shell: sh
        run: |          
          ./gitfirst build --acceptEula false --saveOutputs --tenantDomain $TENANT --refName $GITHUB_REF_NAME --refType $GITHUB_REF_TYPE --sha $GITHUB_SHA --path $GITHUB_WORKSPACE --buildTrigger GitHubAction
      - name: archive-outputs
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: gitfirst-outputs
          path: gitfirst-outputs/**/*

Azure Devops Pipeline Example

The following is an example Azure Devops Pipeline for running a build which will allow you to deploy an app directly from an Azure Devops Git repo into your Qlik Sense Saas cloud tenant.

Note:

You will need to create the variables GF_TENANT, GF_DOWNLOAD_URL, GF_CHECKSUM and GF_ACCEPT_EULA (if you accept the EULA) for your pipeline.
You will need to create the secrets GF_LICENCE (see prerequisites) and GF_TENANT_API_KEY (see here) for your pipeline.
The optional --saveOutputs flag is included as well as an additional PublishBuildArtifacts stage to archive these.
The optional --buildTrigger is also included providing a custom value then available to templating code.

azure-pipelines.yml

trigger:
- master

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    curl --fail $(GF_DOWNLOAD_URL) > gitfirst
    echo "$(GF_CHECKSUM) *gitfirst" | shasum -a 256 --check
    chmod +x ./gitfirst    
  displayName: 'Download Gitfirst'

- script: |
        ./gitfirst build --acceptEula $GF_ACCEPT_EULA --saveOutputs --tenantDomain $GF_TENANT --refName $BUILD_SOURCEBRANCHNAME --refType branch --sha $BUILD_SOURCEVERSION --path $BUILD_SOURCESDIRECTORY --buildTrigger AzureDevOps
  displayName: 'Build Qlik App'
  env:
    LICENCE: $(GF_LICENCE)
    TENANT_API_KEY: $(GF_TENANT_API_KEY)

- task: PublishBuildArtifacts@1
  displayName: 'Attach GitFirst outputs'
  condition: always()
  inputs:
    pathToPublish: $(Build.SourcesDirectory)/gitfirst-outputs
    artifactName: gitfirst-outputs