Recent News
Calling Qliksense Repository API from Apache Drill via sql
Abstract I’ll show how to connect to Qliksense Repository API via sql using Apache Drill.
In this example Qliksense engine service runs at https://qlik.redaelli.org:4242/
Download Download and unzip Apache Drill from https://drill.apache.org/download/
Configure Create or edit the file conf/storage-plugins-override.conf
"storage": { "qliksense" : { "type" : "http", "cacheResults" : true, "connections" : { "p4242" : { "url" : "http://nginx.redaelli.org/qlik/", "method" : "GET", "authType" : "none", "userName" : null, "password" : null, "postBody" : null, "params" : ["filter"], "dataPath" : null, "requireTail" : true, "inputType" : "json", "xmlDataLevel" : 1 } }, "proxyType" : "direct", "enabled" : true } } At the moment Apache drill cannot query external rest services with client certificates DRILL-8052 and so we need a reverse proxy like Nginx.
read more
Calling Talend Cloud Rest API from Apache Drill via sql
Abstract I’ll show how to connect to Talend Cloud API via sql using Apache Drill.
Download Download Apache Drill from https://drill.apache.org/download/
Configure Create or edit the file conf/storage-plugins-override.conf
"storage": { "talendcloud" : { "type" : "http", "cacheResults" : true, "connections" : { "get" : { "url" : "https://api.eu.cloud.talend.com", "method" : "GET", "headers" : { "Authorization" : "Bearer MYSECRETKEY" }, "authType" : "none", "userName" : null, "password" : null, "postBody" : null, "params" : ["query"], "dataPath" : null, "requireTail" : true, "inputType" : "json", "xmlDataLevel" : 1 } }, "proxyType" : "direct", "enabled" : true } } Start Drill Run drill with
read more
Sharing Windows guest vpn connection
Abstract I have a linux workstation with a corporate guest Windows image running on kvm. I would like to share the guest windows vpn connection with my linux host workstation.
My linux host has one network: virbr0 (public) with an ip 192.168.122.x
My vpn/corporate subnet is 10.150.0.0/16
Solution Create on linux/kvm a new host only/private net ( I have an ip 192.168.100.x) and enable it in the windows guest os (ip 192.
read more
Parsing qliksense healthcheck api results
Abstract When you do a stress test/troubleshooting of a qliksense node it is useful to collect the responses of the healthcheck api and extract some useful info from them (which and how many applications were loaded in memory, …)
Collecting data I usually use the command line tool qsense for querying the Qliksense repository
while [ 1 ] do qsense healthcheck qlikhost1.redaelli.org ~/certificates/qlik/client.pem >> healthcheck.jl sleep 60 done Each line of the file healthcheck.
read more
aws-ext
The aws_ext python package contains some useful functions (built on top of boto3) for managing some aws services. At the moment only some utilities for the Aws Glue Data catalog
Installation pip install aws_ext Usage import boto3 import aws_ext session = boto3.session.Session() GLUE from aws_ext import glue_databases glue_client = session.client("glue") Extracting tables with (too) many versions glue_databases.get_tables_with_many_versions(glue_client, database_name="mydb", threshold=1) Deleting old tables versions glue_databases.delete_old_tables_versions(glue_client, database_name="mydb", keep=1, dryrun=True)
read more
dmcommunity.org challenge Jun 2021 with #Prolog
My #prolog solution for “Where is gold?” proposed by dmcommunity.org challenge Jun 2021: 0 means empty, 1 means gold
?- solution(Box1,Box2,Box3). Box1 = Box3, Box3 = 0, Box2 = 1. below the code
:-use_module(library(clpfd)). sentence1( 1,_Box2,_Box3). not_sentence1(0,_Box2,_Box3). sentence2( _Box1,0,_Box3). not_sentence2(_Box1,1,_Box3). sentence3( 0,_Box2,_Box3). not_sentence3(1,_Box2,_Box3). true_only_one_sentence(Box1, Box2, Box3):- ( sentence1(Box1,Box2,Box3), not_sentence2(Box1,Box2,Box3), not_sentence3(Box1,Box2,Box3) ) ; ( not_sentence1(Box1,Box2,Box3), sentence2(Box1,Box2,Box3), not_sentence3(Box1,Box2,Box3) ) ; ( not_sentence1(Box1,Box2,Box3), not_sentence2(Box1,Box2,Box3), sentence3(Box1,Box2,Box3) ). solution(Box1, Box2, Box3):- Box1 in 0..1, /* 0 empty, 1 gold */ Box2 in 0.
read more
GraphQL datasource for Qliksense
Qliksense datasource for GraphQL Apollo server apollo-datasource-qliksense is a Qliksense datasource for the GraphQL Apollo server. “Apollo is the industry-standard GraphQL implementation, providing the data graph layer that connects modern apps to the cloud.”
Installation npm install apollo-datasource-qliksense
Usage Below a very basic sample
File index.js file:
const resolvers = require('./resolvers'); const { ApolloServer, gql } = require('apollo-server'); const { makeExecutableSchema } = require('graphql-tools'); const { QliksenseDataSource } = require('apollo-datasource-qliksense'); const qliksense = new QliksenseDataSource("https://myqlikserver.
read more
qsense python library and comamnd line tool for qliksense
qSense qsense is an python library and command line tool for qliksense
Some useful commands export_remove_old_apps Export (published or passing any other filter) applications to qvd files
qsense export_delete_old_apps qliksense.redaelli.org ~/certificates/client.pem --target_path '/tmp' --modified_days=300 --last_reload_days=300 deallocate_analyzer_licenses_for_professionals Deallocate analyzer license fom users with a professional license
qsense deallocate_analyzer_licenses_for_professionals qliksense.redaelli.org ~/certificates/client.pem --nodryrun delete_removed_exernally_users Delete users that were removed externally (from active directory?)
qsense delete_removed_exernally_users qliksense.redaelli.org ~/certificates/client.pem GROUP --nodryrun
read more
dmcommunity.org challenge Feb 2021 with #Prolog
My #swi #prolog solution for “Benchmark ‘Medical Services’” proposed by dmcommunity.org challenge Feb 2021 can be tested running a public docker image
docker run -p 8888:8888 -d --name matteoredaelli/dmcommunity_org_2021_02:latest wget https://github.com/DMCommunity/dmcommunity_shared/raw/master/MedicalServices.json curl -XPOST -d @MedicalServices.json -H "Accept: application/json" -H "Content-type: application/json" http://localhost:8888/many The decision table is a set of prolog facts like
decision_table('Office','acupuncture','PL123','L','Y','N','2015-01-01','2023-12-31','N','N','N'). decision_table('Outpatient','acupuncture','PL123','L','Y','N','2015-01-01','2023-12-31','N','N','N'). decision_table('Inpatient','acupuncture','PL123','L','N','N','2015-01-01','2023-12-31','N','N','N'). The core of the solution is inside the rules.pl file
parse_input(json([placeOfService=PlaceOfService, type=Type, plan=Plan, groupSize=GroupSize, inNetwork=InNetwork, isCovered=IsCovered, dateOfService=DateOfService, coveredInFull=_, copay=_, coInsurance=_]), json([placeOfService=PlaceOfService, type=Type, plan=Plan, groupSize=GroupSize, inNetwork=InNetwork, isCovered=IsCovered, dateOfService=DateOfService, coveredInFull=CoveredInFull, copay=Copay, coInsurance=CoInsurance])):- decision_table(PlaceOfService, Type, Plan, GroupSize, InNetwork, IsCovered, DateOfService1, DateOfService2, CoveredInFull, Copay, CoInsurance), %% check date: must be between the two dates in decision_table atom_string(DateOfService, DateOfServiceString), atom_string(DateOfService1, DateOfService1String), atom_string(DateOfService2, DateOfService2String), DateOfService1String @=< DateOfServiceString, DateOfServiceString @=< DateOfService2String.
read more
dmcommunity.org challenge Dec 2020 with #Prolog
Below my #swi #prolog solution for “Virtual Chess tournament” proposed by dmcommunity.org challenge Dec 2020
Player 1 is Fischer, player 2 is Kasparov and player 3 is Karpov.
I can find two solutions (Score1,Wins1,Draws1,Loses1 are Fischer’s score, number of wins, draws and loses,..) is:
?- time(solution([[Score1,Wins1,Draws1,Loses1],[Score2,Wins2,Draws2,Loses2],[Score3,Wins3,Draws3,Loses3]])). % 1,297,058 inferences, 0.136 CPU in 0.136 seconds (100% CPU, 9557549 Lips) Score1 = 15, Wins1 = 4, Draws1 = 7, Loses1 = Draws2, Draws2 = 3, Score2 = 13, Wins2 = 5, Loses2 = 6, Score3 = 14, Wins3 = Loses3, Loses3 = 2, Draws3 = 10 ; % 152,142 inferences, 0.
read more