Intro to Lua and Openresty, (another) Interlude: Exploring Lua's `package.path`
Exploring package.path
Every now and again, I run into something like:
/ # lua /src/worker.lua
lua: /src/worker.lua:1: module 'cjson' not found:
no field package.preload['cjson']
no file './cjson.lua'
no file '/usr/local/share/lua/5.1/cjson.lua'
no file '/usr/local/share/lua/5.1/cjson/init.lua'
no file '/usr/local/lib/lua/5.1/cjson.lua'
no file '/usr/local/lib/lua/5.1/cjson/init.lua'
no file '/usr/share/lua/5.1/cjson.lua'
no file '/usr/share/lua/5.1/cjson/init.lua'
no file './cjson.so'
no file '/usr/local/lib/lua/5.1/cjson.so'
no file '/usr/lib/lua/5.1/cjson.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'require'
/src/worker.lua:1: in main chunk
[C]: ?Not able to find cjson? That seems odd, it’s right there:
/ # find / -type f -name cjson*
/usr/local/openresty/lualib/cjson.soHrm, so how do we address this? I had seen a few of these, and I noticed it generally didn’t happen with openresty unless I was requesting a package which I had not yet installed (so that made sense). While I’m not familiar with package management in lua, this seems to be related to that being needed when running outside the pre-configured openresty env I have.
Searching for “lua lib path”, I ended up on http://lua-users.org/wiki/PackagePath, which points out the existence ofpackage.path:
A schematic representation of that list is kept in the variable
package.path. For the above list, that variable contains…
Unfortunately, that wiki doesn’t include an example that updates or customizes package.path.
http://stackoverflow.com/a/4126565 mentions:
You can also change
package.pathin Lua before callingrequire.
..so let’s explore package.path a little:
/ # /usr/local/openresty/luajit/bin/luajit
LuaJIT 2.1.0-beta2 -- Copyright (C) 2005-2016 Mike Pall. http://luajit.org/
JIT: ON SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse
>
> print(package.path)
./?.lua;/usr/local/openresty/luajit/share/luajit-2.1.0-beta2/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/openresty/luajit/share/lua/5.1/?.lua;/usr/local/openresty/luajit/share/lua/5.1/?/init.luaThis looks like a string, let’s see if we can add to it:
> foo = "bar"
> foo += "baz"
stdin:1: '=' expected near '+'
> foo = foo + "baz"
stdin:1: attempt to perform arithmetic on global 'foo' (a string value)
stack traceback:
stdin:1: in main chunk
[C]: at 0x7f9ea8351bd0
> foo = foo .. "baz"
> print(foo)
barbazOK, so use .. for string concatenation (and + requires proper types)
> package.path = package.path .. ";/usr/local/openresty/lualib/?.lua"
> print(package.path)
./?.lua;/usr/local/openresty/luajit/share/luajit-2.1.0-beta2/?.lua;/usr/local/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?/init.lua;/usr/local/openresty/luajit/share/lua/5.1/?.lua;/usr/local/openresty/luajit/share/lua/5.1/?/init.lua;/usr/local/openresty/lualib/?.luaLet’s see if that works in our script..
package.path= package.path .. ";/usr/local/openresty/lualib/?.so"
local cjson = require "cjson"(BTW, there are better ways to do this, see http://stackoverflow.com/a/31626271 for some options)
hrm…
# /usr/local/openresty/luajit/bin/luajit /src/worker.lua
/usr/local/openresty/luajit/bin/luajit: error loading module 'cjson' from file '/usr/local/openresty/lualib/cjson.so':
/usr/local/openresty/lualib/cjson.so:1: unexpected symbol near 'char(127)'
stack traceback:
[C]: at 0x7f7c2a441240
[C]: in function 'require'
/src/worker.lua:2: in main chunk
[C]: at 0x7f7c2a3f7bd0At this point, I’d rather drop back to a “standard/basic lua env” that is not openresty. EG, the openresty env has specifics I don’t understand, and using lua outside of nginx isn’t working right. There is another way to solve this problem: build a basic lua env.
Build stand-alone docker image with lua
EG, for many of the previous exercises, we’ve used the openresty:alpine image, but I sometimes want to use lua straight up, without openresty, luajit, or the environment that openresty has setup (if only to confirm what I see is related to the openresty environment). The module/path issues noted above are also a motivation for this.
Let’s setup a basic docker image with lua and luarocks, based on alpine. We’ll add unzip,curl, gcc, and build-base for fetching/building modules with luarocks:
FROM alpine:latest
RUN apk add --update unzip curl build-base gcc lua5.1 lua5.1-dev luarocks5.1
VOLUME /src
WORKDIR /src
ENTRYPOINT /bin/shHere is our Makefile, to make this a little easier:
build:
docker build --tag=lua --rm .
run:
docker run -it --rm -v `pwd`:/src lua
test:
# Run the following two commands:
# make run
# luarocks-5.1 install lua-cjson && lua5.1 cjson-test.luaWe will mount the current working directory as /src in the image, and run our lua from there.
Go to examples/stand-alone-lua-image and build it:
ᐅ cd examples/stand-alone-lua-image
ᐅ make build
docker build --tag=lua --rm .
Sending build context to Docker daemon 4.096 kB
Step 1 : FROM alpine:latest
---> fe3e188d9166
Step 2 : RUN apk add --update unzip curl build-base gcc lua5.1 lua5.1-dev luarocks5.1
---> Running in 73bbb2c31559
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.5/community/x86_64/APKINDEX.tar.gz
(1/29) Upgrading musl (1.1.15-r5 -> 1.1.15-r6)
(2/29) Installing binutils-libs (2.27-r0)
(3/29) Installing binutils (2.27-r0)
(4/29) Installing gmp (6.1.1-r0)
(5/29) Installing isl (0.17.1-r0)
(6/29) Installing libgomp (6.2.1-r1)
(7/29) Installing libatomic (6.2.1-r1)
(8/29) Installing pkgconf (1.0.2-r0)
(9/29) Installing libgcc (6.2.1-r1)
(10/29) Installing mpfr3 (3.1.5-r0)
(11/29) Installing mpc1 (1.0.3-r0)
(12/29) Installing libstdc++ (6.2.1-r1)
(13/29) Installing gcc (6.2.1-r1)
(14/29) Installing make (4.2.1-r0)
(15/29) Installing musl-dev (1.1.15-r6)
(16/29) Installing libc-dev (0.7-r1)
(17/29) Installing fortify-headers (0.8-r0)
(18/29) Installing g++ (6.2.1-r1)
(19/29) Installing build-base (0.4-r1)
(20/29) Installing ca-certificates (20161130-r0)
(21/29) Installing libssh2 (1.7.0-r2)
(22/29) Installing libcurl (7.52.1-r2)
(23/29) Installing curl (7.52.1-r2)
(24/29) Upgrading musl-utils (1.1.15-r5 -> 1.1.15-r6)
(25/29) Installing lua5.1-libs (5.1.5-r2)
(26/29) Installing lua5.1 (5.1.5-r2)
(27/29) Installing lua5.1-dev (5.1.5-r2)
(28/29) Installing luarocks5.1 (2.4.2-r0)
(29/29) Installing unzip (6.0-r2)
Executing busybox-1.25.1-r0.trigger
Executing ca-certificates-20161130-r0.trigger
OK: 161 MiB in 38 packages
---> 6790dd1671c6
Removing intermediate container 73bbb2c31559
Step 3 : VOLUME /src
---> Running in 24a551c29f23
---> 61dc8c8fe791
Removing intermediate container 24a551c29f23
Step 4 : WORKDIR /src
---> Running in f5d7b97948a7
---> 1ac18b1d38f1
Removing intermediate container f5d7b97948a7
Step 5 : ENTRYPOINT /bin/sh
---> Running in 1ceba58c9546
---> ef633a63546e
Removing intermediate container 1ceba58c9546
Successfully built ef633a63546eLet’s test the image with our cjson situation above. We have a simple script - cjson-test.lua - it will exit 0 if it can import and use the cjson package:
local cjson = require "cjson"
print(cjson.encode({status = "success!"}))Let’s use the lua base image we created above, install the lua-cjson package, and then run our cjson-test.lua script:
ᐅ make run
docker run -it --rm -v `pwd`:/src lua
/src #
/src # luarocks-5.1 install lua-cjson
Installing https://luarocks.org/lua-cjson-2.1.0-1.src.rock
gcc -O2 -fPIC -I/usr/include -c lua_cjson.c -o lua_cjson.o
In file included from lua_cjson.c:47:0:
fpconv.h:15:20: warning: inline function 'fpconv_init' declared but never defined
extern inline void fpconv_init();
^~~~~~~~~~~
gcc -O2 -fPIC -I/usr/include -c strbuf.c -o strbuf.o
gcc -O2 -fPIC -I/usr/include -c fpconv.c -o fpconv.o
gcc -shared -o cjson.so -L/usr/lib lua_cjson.o strbuf.o fpconv.o
No existing manifest. Attempting to rebuild...
lua-cjson 2.1.0-1 is now installed in /usr/local (license: MIT)Awesome, installed the package. Let’s run the test:
/src # lua5.1 cjson-test.lua
{"status":"success!"}
