Hi there 馃憢

Welcome to my blog

Calling Qliksense Repository API from Apache Drill via sql

Abstract I鈥檒l show how to connect to Qliksense Repository API via sql using Apache Drill. In this example Qliksense engine service runs at https://qlik.redaelli.org:4242/ Download Download and unzip Apache Drill from https://drill.apache.org/download/ Configure Create or edit the file conf/storage-plugins-override.conf "storage": { "qliksense" : { "type" : "http", "cacheResults" : true, "connections" : { "p4242" : { "url" : "http://nginx.redaelli.org/qlik/", "method" : "GET", "authType" : "none", "userName" : null, "password" : null, "postBody" : null, "params" : ["filter"], "dataPath" : null, "requireTail" : true, "inputType" : "json", "xmlDataLevel" : 1 } }, "proxyType" : "direct", "enabled" : true } } At the moment Apache drill cannot query external rest services with client certificates DRILL-8052 and so we need a reverse proxy like Nginx....

February 23, 2022 路 2 min 路 271 words 路 Matteo Redaelli

Calling Talend Cloud Rest API from Apache Drill via sql

Abstract I鈥檒l show how to connect to Talend Cloud API via sql using Apache Drill. Download Download Apache Drill from https://drill.apache.org/download/ Configure Create or edit the file conf/storage-plugins-override.conf "storage": { "talendcloud" : { "type" : "http", "cacheResults" : true, "connections" : { "get" : { "url" : "https://api.eu.cloud.talend.com", "method" : "GET", "headers" : { "Authorization" : "Bearer MYSECRETKEY" }, "authType" : "none", "userName" : null, "password" : null, "postBody" : null, "params" : ["query"], "dataPath" : null, "requireTail" : true, "inputType" : "json", "xmlDataLevel" : 1 } }, "proxyType" : "direct", "enabled" : true } } Start Drill Run drill with...

February 4, 2022 路 2 min 路 292 words 路 Matteo Redaelli

Sharing Windows guest vpn connection

Abstract I have a linux workstation with a corporate guest Windows image running on kvm. I would like to share the guest windows vpn connection with my linux host workstation. My linux host has one network: virbr0 (public) with an ip 192.168.122.x My vpn/corporate subnet is 10.150.0.0/16 Solution Create on linux/kvm a new host only/private net ( I have an ip 192.168.100.x) and enable it in the windows guest os (ip 192....

October 3, 2021 路 2 min 路 290 words 路 Matteo Redaelli

Parsing qliksense healthcheck api results

Abstract When you do a stress test/troubleshooting of a qliksense node it is useful to collect the responses of the healthcheck api and extract some useful info from them (which and how many applications were loaded in memory, 鈥) Collecting data I usually use the command line tool qsense for querying the Qliksense repository while [ 1 ] do qsense healthcheck qlikhost1.redaelli.org ~/certificates/qlik/client.pem >> healthcheck.jl sleep 60 done Each line of the file healthcheck....

September 15, 2021 路 3 min 路 445 words 路 Matteo Redaelli

aws-ext

The aws_ext python package contains some useful functions (built on top of boto3) for managing some aws services. At the moment only some utilities for the Aws Glue Data catalog Installation pip install aws_ext Usage import boto3 import aws_ext session = boto3.session.Session() GLUE from aws_ext import glue_databases glue_client = session.client("glue") Extracting tables with (too) many versions glue_databases.get_tables_with_many_versions(glue_client, database_name="mydb", threshold=1) Deleting old tables versions glue_databases.delete_old_tables_versions(glue_client, database_name="mydb", keep=1, dryrun=True)

June 25, 2021 路 1 min 路 67 words 路 Matteo Redaelli