Merge Pandas DataFrame with Nested Dictionary

Not an avid Pandas or Numpy user myself, but I had to spend some time lately to optimise probably a fairly common process: looking up a nested dictionary (2 or more levels) to find the right values element-wise for a column in a Pandas DataFrame. If it’s not clear, the problem I’m trying to solve here is to optimise a look-up function that can be done with .apply() to something more performant.

You might say, why not using .map()? Because the look-up function is not y = f(x), no, it is more like y = f(x, a) or even y = f(x, a, b), depending on the level of nestedness.

As mentioned earlier, this can be implemented with .apply() by supplying a Python function that does the look-up. However, .apply() is very slow (it’s not vectorised). The solution here is actually straightforward (I’m very new to Pandas and it took me some time to get here so I decided to make a note here for this). We first flatten the nested dictionary to have different levels of keys as tuples, which allows us to create a DataFrame with MultiIndex. With MultiIndex, we can easily apply .merge to join the DataFrame objects.

Hopefully the code snippet is more understandable.

import pandas as pd

nested_dict = {
    "A": {
        "Apple": "Red",
        "Banana": "Green",
    "B": {"Apple": "Green", "Banana": "Yellow"},
df = pd.DataFrame.from_dict(
        "Fruit": {0: "Apple", 1: "Banana", 2: "Banana"},
        "Price": {0: 0.911, 1: 1.734, 2: 1.844},
        "Bucket": {0: "A", 1: "B", 2: "A"},

# Method 1: .apply()
# Apply Python function element-wise, as slow as a regular for loop
df1 = df.copy()
df1["Color"] = df1.apply(
    lambda row: nested_dict.get(row["Bucket"], {}).get(row["Fruit"]), axis=1

# Method 2: .merge()
# Vectorized, much faster (even though the big O is the same)
df2 = df.copy()
# The only overhead is to construct a dataframe with 'MultiIndex'
nested_df = pd.DataFrame.from_dict(
        (inner_key, outer_key): value
        for outer_key, outer_value in nested_dict.items()
        for inner_key, value in outer_value.items()
nested_df.index = pd.MultiIndex.from_tuples(nested_df.index)
nested_df.rename(columns={0: "Color"}, inplace=True)
df2 = df2.merge(nested_df, how="left", left_on=("Fruit", "Bucket"), right_index=True)

Windows 10/11 拼音输入法英式键盘






按下Windows键(或者点击开始菜单按钮)输入 regedit,回车或者点击注册表编辑器。进入 HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layouts 然后一个一个找(可以使用方向键翻,用鼠标点可能会点疯),找到你要修改的语言,例如这里我要改的是简体中文,文件夹00000804对应的就是简体中文的键盘布局 (Layout Text 的值是 Chinese (Simplified) - US Keyboard),双击 Layout File 把值从 KBDUS.DLL 改成 KBDUK.DLL,可以顺便把 Layout Text 的值里面的 US Keyboard 改成 UK Keyboard。修改完成后需要重启电脑(或者登出再登入)生效。

如果你使用的是其他键盘布局,可以挨个找到该原生语言的布局(例如如果你用的是美式 Dvorak 键盘,挨个翻,可以找到 00010409 里面的 Layout Text 写的是 United States-Dvorak),找到正确的 Layout File,然后把中文的 Layout File 修改成对应的值(例如美式 Dvorak 就会是 KBDDV.DLL)。





本方法对Windows 11同样有效,但是从Windows 10升级到Windows 11后相关注册表值会被重置,按照上面的内容再修改一次就可以了。

Building KDE Frameworks on Windows from Source

Some notes on how to build KDE Frameworks packages from source on Windows using Visual Studio tools.

To do so, you need to first have a version of Qt compiled by MSVC installed. Some system environment variables to be set, using Qt 5.15.2 as an example:

  • PATH needs to add C:\Qt\5.15.2\msvc2019_64\bin
  • Qt_DIR needs to be set to C:\Qt\5.15.2\msvc2019_64

Example instructions for building CMake-based projects (all KDE projects), the command below should be executed in x64 Native Tools Command Prompt.

mkdir build && cd build
cmake .. -G "NMake Makefiles" -DCMAKE_INSTALL_PREFIX="C:\Qt\5.15.2\msvc2019_64"
nmake && nmake install

This will install the compiled KDE module into the Qt installation path. You can install it elsewhere, but if you do, make sure you update PATH environment variable accordingly.

A New Termux Mirror

TL; DR. is a new Termux packages mirror! Maintained by me, synchronised every six hours, located in the United Kingdom, hosted by Oracle Cloud.

In the full article below, I’ll write up how to set up a Termux mirror (or in general, a Debian packages repository mirror).

Continue reading “A New Termux Mirror”

Building Qt for Termux Android

This is by all means, not the first blog post about Termux. It serves as a journal for myself, as well for anyone who’s interested in cross building Qt or Qt-related projects for Termux (which is a native Linux environment for Android) from their x86_64 machines.

We already have @xeffyr who has done a great amount of work on building Qt for Termux. The work I’ve done recently would be 10 times harder, if not for what’s achieved by them already.

If you’re new to cross building for Termux, I recommend you to start with Developer Wiki.

Continue reading “Building Qt for Termux Android”

调教三星 DeX 的输入法

不知道有多少人是被 DeX 吸引到三星 Galaxy S 系列中的,DeX 可是其中一个很大的卖点(至少在欧洲和北美市场, DeX 是重点宣传的功能)。连上显示器(现在支援无线 DeX 了),或者平板切换开启 DeX 模式,马上生产力上升了好几倍!

然后马上你就会发现输入法被锁死在三星键盘上了。无论你本来喜爱的是谷歌的 GBoard 还是搜狗拼音,都被三星键盘替代了(退出 DeX 后输入法会变回去)。

三星键盘的中文录入是勉强能用级别,明显的卡顿和超过10个候选词的界面让你觉得这软件就没有 QA 的。搜索了一大圈网络教程,亲测使用 adb 可以修改 DeX 的输入法(Android 11, One UI 3.1)。

比如更改成谷歌 GBoard :

adb shell ime set

当然如果你的设备已经外接到显示器和键鼠了,又怎么通过 PC 或 Mac 来执行 adb 指令呢?好消息是 Android 开发者选项现在打开 WiFi adb debugging 了!更好的消息是你可以安装 Termux 然后下载为 Termux 预编译好的 adb 二进制程序,直接从手机/平板运行 adb 来改输入法。

另一个反人类的默认设置就是键盘候选框会显示在手机(或者平板电脑)上,这在打字的时候要去侧头看另一边来输入汉字简直莫名其妙。好在解决这个问题有个很简单的设置项(埋在了Samsung DeX里)。

设置 -> Samsung DeX -> 键盘 -> 屏幕键盘位置, 改成“电视或显示器”

写到这里, DeX 的中文输入依然不好用(稍微能用一点了),在普通 Android 界面下能够追随光标的 GBoard 在 DeX 下也不能跟随光标了。而且输入习惯和 PC 或 Mac 上的输入法有所区别,也还是只能算勉强能用级别了。

Fedora Linux with MATE Desktop on Android

I know some of you geeks have already done something like that, running a GNU/Linux distribution on top of an Android device. It’s an interesting time, with projects like proot we don’t need root privilege on our Android devices to run a containerized Linux environment. Here in this post, I’ve written down the steps that I’ve taken to have a usable desktop environment up running on my Galaxy Tab S5e (this blog post has been written up purely within such environment).

Continue reading “Fedora Linux with MATE Desktop on Android”


自从几年前卖掉华硕的一台小Chromebook后,看到Android和Linux应用程式的支持增加,又开始有点馋涎Chromebook……不过因为今年疫情全球大爆发,电子产品(尤其是适合居家办公和学习的)价格一路上扬(或者缺货)。Chromebook今年出货量增长更是迅猛(Windows PC, MacBook, Chromebook分类中增长幅度最大)。问我这台2015年买的老笔电尚能饭否?答曰:Brunch

出厂预装的Windows 10早被我卸载干净了,单Linux系统(OpenSUSE)用了有一年多了,看到Brunch这个项目能在自己笔电上体验Chrome OS变身Chromebook,赶紧试验了一把,没想到效果出奇地好!这里特意用中文记载一下Linux下安装Chrome OS和后续升级的办法以供中文网络世界的朋友参考。

Continue reading “来份Brunch笔电瞬间变身Chromebook”

Building Linux Kernel on Odroid-U3

This is the year 2020, Odroid-U3 is far from a powerful ARM development board in today’s standard, but it is still more than capable! Quad-core ARMv7 CPU and 2GB RAM, which means I can still run some light services and test my toy projects without paying any fees to AWS or Azure. The year 2020 also means that this little device can be powered by the mainline Linux kernel without many troubles (if any)! Better than that, the process is ridiculously straightforward!

In this post, I put together the steps I did to build the Linux kernel using upstream mainline source code natively on Odroid-U3. If you’re interested in cross-building from your x86 machines, you can find plenty of tutorials on that on the Internet.

Continue reading “Building Linux Kernel on Odroid-U3”