Enabling Battle.net OAuth2 in python-social-auth

Having a running setup, using OAuth2 login with Django & python-social-auth==0.1.19, you can simply create a file bnet.py within the social/backends folder:

from social.backends.oauth import BaseOAuth2
class BnetOAuth2(BaseOAuth2):
    """Battlenet OAuth2 authentication backend"""
    name = 'bnet-oauth2'
    EMAIL_SUFFIX = "code-corner.de"
    AUTHORIZATION_URL = 'https://eu.battle.net/oauth/authorize'
    ACCESS_TOKEN_URL = 'https://eu.battle.net/oauth/token'
    DEFAULT_SCOPE = ['wow.profile',
    EXTRA_DATA = [
        ('refresh_token', 'refresh_token', True),
        ('expires_in', 'expires'),
        ('token_type', 'token_type', True)
    def asd_get_user_id(self, details, response):
        if self.setting('USE_UNIQUE_USER_ID', False):
            return response['email']
            return response['username']
    def get_user_details(self, response):
        return {'username': response.get('username', ''),
                'email': response.get('email', ''),
                'fullname': response.get('name', ''),
                'first_name': response.get('given_name', ''),
                'last_name': response.get('family_name', '')}
    def user_data(self, access_token, *args, **kwargs):
        id = self.get_json(
            params={'access_token': access_token}
        btag = self.get_json(
            params={'access_token': access_token}
        first_name, last_name = btag.split("#")
        return {"id": id,
                "username": first_name,
                "email": "%d@%s" % (id, self.EMAIL_SUFFIX),
                "family_name": last_name,
                "given_name": first_name,
                "name": btag}

Finally activate the created backend and don’t forget to add the BNet API keys to your django settings. Now go and try the login.

AUTHENTICATION_BACKENDS = ('social.backends.bnet.BnetOAuth2')

After login, a new user should be created using the part of the battle tag before the # as username and first_name and the number behind the # as last_name. The email will be filled using the BNet account id and a user-defined suffix.

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Dual GPU notebook configuration Intel + AMD/ATI

Add catalyst + xorg114 pacman repos to your /etc/pacman.conf:

Server = http://catalyst.wirephire.com/repo/catalyst/$arch
Server = http://catalyst.wirephire.com/repo/xorg114/$arch

Install required xorg server and driver packages:


Create /etc/X11/xorg.conf using aticonfig –initial

Section "ServerLayout"
        Identifier     "aticonfig Layout"
        Screen      0  "aticonfig-Screen[0]-0" 0 0
Section "Module"
Section "Monitor"
        Identifier   "aticonfig-Monitor[0]-0"
        Option      "VendorName" "ATI Proprietary Driver"
        Option      "ModelName" "Generic Autodetecting Monitor"
        Option      "DPMS" "true"
Section "Device"
        Identifier  "aticonfig-Device[0]-0"
        Driver      "fglrx"
        BusID       "PCI:1:0:0"
Section "Screen"
        Identifier "aticonfig-Screen[0]-0"
        Device     "aticonfig-Device[0]-0"
        Monitor    "aticonfig-Monitor[0]-0"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24

Switch gpu using the AMD tool: amdcccle

Direct rendering works: glxinfo | grep direct

direct rendering: Yes


If glxgears / glxinfo gives errors like:

libGL error: failed to load driver: i965
libGL error: Try again with LIBGL_DEBUG=verbose for more details.
libGL error: failed to load driver: swrast
libGL error: Try again with LIBGL_DEBUG=verbose for more details.

… set the catalyst libGL path:

export LIBGL_DRIVERS_PATH=/usr/lib/catalystpxp/fglrx/
export LD_PRELOAD=/usr/lib/catalystpxp/fglrx/fglrx-libGL.so.1.2

OpenCL works: clinfo

Number of platforms:                             1
  Platform Profile:                              FULL_PROFILE
  Platform Version:                              OpenCL 1.2 AMD-APP (1348.4)
  Platform Name:                                 AMD Accelerated Parallel Processing
  Platform Vendor:                               Advanced Micro Devices, Inc.
  Platform Extensions:                           cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
  Platform Name:                                 AMD Accelerated Parallel Processing
Number of devices:                               2
  Device Type:                                   CL_DEVICE_TYPE_GPU
  Device ID:                                     4098
  Board name:                                    AMD Radeon HD 8600/8700M
  Device Topology:                               PCI[ B#1, D#0, F#0 ]
  Max compute units:                             6
  Max work items dimensions:                     3
    Max work items[0]:                           256
    Max work items[1]:                           256
    Max work items[2]:                           256
  Max work group size:                           256
  Preferred vector width char:                   4
  Preferred vector width short:                  2
  Preferred vector width int:                    1
  Preferred vector width long:                   1
  Preferred vector width float:                  1
  Preferred vector width double:                 1
  Native vector width char:                      4
  Native vector width short:                     2
  Native vector width int:                       1
  Native vector width long:                      1
  Native vector width float:                     1
  Native vector width double:                    1
  Max clock frequency:                           825Mhz
  Address bits:                                  32
  Max memory allocation:                         1073741824
  Image support:                                 Yes
  Max number of images read arguments:           128
  Max number of images write arguments:          8
  Max image 2D width:                            16384
  Max image 2D height:                           16384
  Max image 3D width:                            2048
  Max image 3D height:                           2048
  Max image 3D depth:                            2048
  Max samplers within kernel:                    16
  Max size of kernel argument:                   1024
  Alignment (bits) of base address:              2048
  Minimum alignment (bytes) for any datatype:    128
  Single precision floating point capability
    Denorms:                                     No
    Quiet NaNs:                                  Yes
    Round to nearest even:                       Yes
    Round to zero:                               Yes
    Round to +ve and infinity:                   Yes
    IEEE754-2008 fused multiply-add:             Yes
  Cache type:                                    Read/Write
  Cache line size:                               64
  Cache size:                                    16384
  Global memory size:                            2009071616
  Constant buffer size:                          65536
  Max number of constant args:                   8
  Local memory type:                             Scratchpad
  Local memory size:                             32768
  Kernel Preferred work group size multiple:     64
  Error correction support:                      0
  Unified memory for Host and Device:            0
  Profiling timer resolution:                    1
  Device endianess:                              Little
  Available:                                     Yes
  Compiler available:                            Yes
  Execution capabilities:                                
    Execute OpenCL kernels:                      Yes
    Execute native function:                     No
  Queue properties:                              
    Out-of-Order:                                No
    Profiling :                                  Yes
  Platform ID:                                   0x00007f33cbfb54c0
  Name:                                          Oland
  Vendor:                                        Advanced Micro Devices, Inc.
  Device OpenCL C version:                       OpenCL C 1.2 
  Driver version:                                1348.4 (VM)
  Profile:                                       FULL_PROFILE
  Version:                                       OpenCL 1.2 AMD-APP (1348.4)
  Extensions:                                    cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_atomic_counters_32 cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer

DOGE mining also works with cgminer-gpu from AUR with 55Kh/s at intensity 10:

cgminer version 3.7.2 - Started: [2014-01-16 18:39:45]
 (5s):54.25K (avg):48.66Kh/s | A:64  R:0  HW:0  WU:42.1/m
 ST: 2  SS: 0  NB: 3  LW: 19  GF: 0  RF: 0
 Connected to xxx diff 16 with stratum as user xxx
 Block: daa10059...  Diff:505  Started: [18:39:58]  Best share: 97
 [P]ool management [G]PU management [S]ettings [D]isplay options [Q]uit
 GPU 0:                | 54.25K/48.66Kh/s | A:64 R:0 HW:0 WU: 42.1/m I:10
 [2014-01-16 18:39:44] Started cgminer 3.7.2
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

AMD OpenCL on arch linux

Due to an old AMD Radeon HD 4890 graphics card, I was forced to use the additional repos xorg112 and catalyst-hd234k to get a working AMD legacy fglrx / X server duo. Till now, clinfo runs fine. However, in order to compile OpenCL applications I installed the amdapp-sdk package from AUR. Afterwards, clinfo and every other OpenCL binary segfaulted instantaneously due to mixed OpenCL versions. Replacing /opt/AMDAPP/SDK/lib/libamdocl64.so with /usr/lib/libamdocl64.so solved this issue.

catalyst-utils: /usr/lib/libamdocl64.so
amdapp-sdk: /opt/AMDAPP/SDK/lib/libamdocl64.so

The GPU load can be monitored using the following command:

aticonfig --adapter=0 --od-getclocks
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Python-social-auth requires real hostname when using apache+gunicorn

After a successful authentication using google-oauth2 with the python-social-auth tool, the client gets redirected to a ../complete/ page. However, after forwarding all non-static traffic from the apache to a local gunicorn server, the redirection url points to http://localhost:8000/complete rather than to the real website. This can be easily fixed by enabling the ProxyPreserveHost setting in the apache config:

<VirtualHost *:80>
        ServerAdmin ...
        ServerName ...
        Alias /static /.../static
        DocumentRoot ...
        <Directory ...>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        ProxyPreserveHost on
        RewriteEngine on
        RewriteRule ^/static/.* - [L]
        # Django is run via gunicorn. So proxy the rest.
        RewriteRule ^(.*) http://localhost:8000$1 [P]

VN:F [1.9.22_1171]
Rating: 5.0/10 (1 vote cast)

Supervisord at 100% load for gunicorn

Starting a gunicorn django server with the supervisord daemon using the deprecated gunicorn_django command works but results in 100% load of the supervisor process.

process_name=gunicorn  ; process_name expr (default %(program_name)s)
numprocs=1                    ; number of processes copies to start (def 1)
directory=/home/gunicorn  ; directory to cwd to before exec (def no cwd)
autostart=true                ; start at supervisord start (default: true)
autorestart=true        ; whether/when to restart (default: unexpected)
user=gunicorn                   ; setuid to this UNIX account to run the program
redirect_stderr=true          ; redirect proc stderr to stdout (default false)

After adding gunicorn to the INSTALLED_APPS in the project settings, the gunicorn server can be easily started with the python manage.py run_gunicorn command and everything works fine.

VN:F [1.9.22_1171]
Rating: 6.0/10 (1 vote cast)

streaMplayer – play flash streams with mplayer

Want to watch a flash live stream but your browser is not able to play the stream smoothly ? Or maybe you simply hate flash movies… Here is the solution ! Catch the stream and pipe it to mplayer. Here is small python script to set the required iptables and start mplayer afterwards.

Usage: Start the script; then open / reload the website that includes the flash player in your browser.

import time
import subprocess
retries = 3
sleep = 3
iptAdd = "sudo iptables -t nat -A OUTPUT -p tcp --dport 1935 -j REDIRECT"
iptDel = "sudo iptables -t nat -D OUTPUT -p tcp --dport 1935 -j REDIRECT"
iptc = subprocess.Popen(iptAdd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
_, stderr = iptc.communicate()
if not len(stderr) == 0:
    rtmpsrv = subprocess.Popen("rtmpsrv", stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=0)
    while True:
        line = rtmpsrv.stderr.readline()
        if line.find(b"Closing connection") >= 0:
    stdout, _ = rtmpsrv.communicate()
    rtmpdump = b"".join(stdout.split(b"\n"))
    rtmpdump = None
iptc = subprocess.Popen(iptDel, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
_, stderr = iptc.communicate()
if not len(stderr) == 0:
    print("Could not detect stream.")
if not rtmpdump is None:
    rtmpdump = rtmpdump.decode("ASCII")
    print("Found rtmpdump command: '%s'\n" % rtmpdump)
    mp = "%s | mplayer -" % rtmpdump[:rtmpdump.find(" -o \"")]
    print("Executing: '%s'\n" % mp)
    retry = 0
    while True:
        mplayer = subprocess.Popen(mp, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        _, stderr = mplayer.communicate()
        if stderr.find(b"ERROR") >= 0:
            print("Error :(")
            if retry < retries:
                retry += 1
                print("Retry (%d)" % retry)
                print("Giving up")




VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)

Faster file IO in python using cython

Reading large files in Python sometimes feels incredible slow. Here are some approaches using Cython to minimize reading times. Simply compiling the existing python code with Cython reduces the reading times by 23%. By introducing explicit type definitions, I could finally reach C++ reading speeds wich are 4.4x faster than pure Python code. However, when I used the generator keyword yield to iterate over all lines in an external Python function without exploiting my memory, the required runtime doubles for this approach. The used code snippets are listed below.


File: file_io_python.py Simple python function to read in a file line by line.

def read_file_python(filename):
    f = open(filename, "rb")
    while True:
        line = f.readline()
        if not line: break
        #yield line
    return []

File: file_io.pyx Cython file, containing a pure python function and a cython optimized function for linewise file reading.

from libc.stdio cimport *
cdef extern from "stdio.h":
    #FILE * fopen ( const char * filename, const char * mode )
    FILE *fopen(const char *, const char *)
    #int fclose ( FILE * stream )
    int fclose(FILE *)
    #ssize_t getline(char **lineptr, size_t *n, FILE *stream);
    ssize_t getline(char **, size_t *, FILE *)
def read_file_slow(filename):
    f = open(filename, "rb")
    while True:
        line = f.readline()
        if not line: break
        #yield line
    return []
def read_file(filename):
    filename_byte_string = filename.encode("UTF-8")
    cdef char* fname = filename_byte_string
    cdef FILE* cfile
    cfile = fopen(fname, "rb")
    if cfile == NULL:
        raise FileNotFoundError(2, "No such file or directory: '%s'" % filename)
    cdef char * line = NULL
    cdef size_t l = 0
    cdef ssize_t read
    while True:
        read = getline(&line, &l, cfile)
        if read == -1: break
        #yield line
    return []

File: file_io.cppComparison code for C++.

#include "stdio.h"
#include <stdlib.h>
int main()
    FILE* cfile = fopen("trajectory.pdb", "rb");
    if(cfile == NULL) return 1;
    char * line = NULL;
    size_t l = 0;
    ssize_t read;
        read = getline(&line, &l, cfile);
        if(read == -1) break;
    return 0;

File: file_io_bench.pyPython code to test and benchmark all different functions.
import timeit
count = 10
check = False
if check:
    from file_io import read_file, read_file_slow
    import hashlib
    m = hashlib.new("md5")
    for line in read_file_slow("trajectory.pdb"):
    h1 = m.hexdigest()
    m = hashlib.new("md5")
    for line in read_file("trajectory.pdb"):
    h2 = m.hexdigest()
    assert h1 == h2, Exception("read error")
    print("read functions: ok")
t = timeit.Timer("""for line in read_file_python("trajectory.pdb"):
  pass""", """from file_io_python import read_file_python""")
t1 = t.timeit(count)
print("Python", t1, "sec")
t = timeit.Timer("""for line in read_file_slow("trajectory.pdb"):
  pass""", """from file_io import read_file_slow""")
t2 = t.timeit(count)
print("Cython", t2, "sec")
t = timeit.Timer("""for line in read_file("trajectory.pdb"):
  pass""", """from file_io import read_file""")
t3 = t.timeit(count)
print("cdef Cython", t3, "sec")
t = timeit.Timer("""s = subprocess.Popen("./a.out", shell=True)
""", """import subprocess""")
t4 = t.timeit(count)
print("C", t4, "sec")

VN:F [1.9.22_1171]
Rating: 6.6/10 (5 votes cast)

Logitech mouse not working after kernel upgrade to 3.10.2-1-ARCH

After upgrading my kernel to 3.10.2-1-ARCH, my logitech wireless mouse m325 stoped working.
I installed ltunify-git from AUR repo to control the logitech unifying device. After unpairing and pairing ‘idx=1 Mouse M325’, the mouse was working again.

sudo ltunify -d /dev/hidraw0 unpair 1
sudo ltunify -d /dev/hidraw0 pair 10

Now turn your mouse off and on again ! (and press a button)

VN:F [1.9.22_1171]
Rating: 6.5/10 (2 votes cast)

Decrypting RSDF files

A small code-snippet to decrypt links from a RSDF container:

def decryptRSDF(filename):
    from Crypto.Cipher import AES

    links = []

    f = open(filename, "r")
    lines = f.readlines()

    data = bytearray.fromhex("".join(lines))
    array = data.split("\n")

    key = bytearray.fromhex("8C35192D964DC3182C6F84F3252239EB4A320D2500000000")
    iv = bytearray.fromhex("a3d5a33cb95ac1f5cbdb1ad25cb0a7aa")

    aes_context = AES.new(str(key), AES.MODE_ECB, str(iv))

    for line in array:
        url_in = base64.b64decode(line)
        length = len(url_in)

        if length > 0:
            url_input = bytearray(url_in)
            url_output = bytearray(length)

            #1 byte
            output_block = bytearray(aes_context.encrypt(str(iv)))

            url_output[0] = url_input[0] ^ output_block[0]

            #other bytes
            for n in range(1, length+1):
                iv[:15] = iv[1:]
                iv[15] = url_input[n-1]

                if n < length:
                    output_block = bytearray(aes_context.encrypt(str(iv)))
                    url_output[n] = url_input[n] ^ output_block[0]


    return links
VN:F [1.9.22_1171]
Rating: 3.0/10 (1 vote cast)

PyQt4 QFileDialog freezes when qt4reactor is running

To use the twisted reactor inside a Qt gui application, I’m using the qt4reactor package:

app = QtGui.QApplication(sys.argv)

import qt4reactor

from twisted.internet import reactor
factory = Factory()

gui = Gui(app)



But I realized that after the “reactor.runReturn()” line, each QFileDialog freezes the complete program.

filename = QtGui.QFileDialog.getOpenFileName()

A solution is to use the non-native dialog instead:

filename = QtGui.QFileDialog.getOpenFileName(options=QtGui.QFileDialog.DontUseNativeDialog)
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)