Using the Idle Interpreters and Compiler
Idle scripts can be executed in two fundamentally different ways:
- through one of the two Idle interpreters (idle.exe for console-based scripts and idlew.exe for GUI scripts). This is as close to instant gratification as Idle will get you, because the script is loaded, compiled (in memory) to a byte code chunk which is then executed by Idle's virtual machine (if the script had no errors, that is: otherwise the interpreter will greet you with a tidy error message). Both interpreters can also load and execute pre-compiled byte code chunks (.idol files) produced by the Idle compiler. This saves some overhead which might become important if you are executing truly huge scripts.
- through the Idle compiler (idlec.exe). The compiler will translate a script either to an Idle byte code file (which can later be loaded and executed by the console or GUI interpreter) or to an executable file (which can be started separately). If all modules required by a script are compiled into the executable (the compiler will do that), then the only additional file needed to run it is the Idle runtime DLL (ie idle03.dll). If your script uses any of the extensions, like the Perl interface or the SQLite3 module, you will of course need those DLLs as well.
Note: the Idle just-in-time (JIT) compiler kicks in automatically whenever a byte code chunk is executed by the Idle virtual machine. This happens transparently in the background and normally requires no further interaction by the user.
The Idle interpreter for console-mode scripts (command-line-based, idle.exe)
top of page
This version of the interpreter is the one to use for console-based scripts. It always runs in a console window and will handle its input and output via this window (as long as there was no explicit redirection on the command line). However, console-based scripts can use GUI functionality by simply require( )ing the gui module:
require('gui')
The idle.exe interpreter can be started with the following command line options:
usage: idle [options] [scriptfile [args]] Available options are: -e stat execute string 'stat' -i enter interactive mode after executing 'scriptfile' -l name require library 'name' -m disable token-filter-based macros -J cmd perform JIT control command -O[lev] set JIT optimization level [0,1,2] -v show version information -- stop handling options - execute stdin and stop handling options
The options have the following meaning:
- -e: execute: immediately executes the given string as Idle code. You will have to quote the string if it contains whitespace or other special characters.
- -i: interactive: enters interactive mode, after loading libraries and running any scriptfile given on the command line
- -l: load library: does the equivalent of require('name')
- -m: disables the internal token-filter-based macro processor
- -J: JIT compiler control: calls the jit module with the given command (on, off, ...)
- -O: optimisation level: sets the JIT optimisation level to a value between 0 (no optimisation) and 2 (full optimisation).
- -v: version: prints a version information string
After handling the options, idle runs the scriptfile, if given, passing any args as string arguments (see below how an Idle script accesses its command line arguments). When called without arguments, idle behaves as 'idle -i' when the standard input (stdin) is a console, and as 'idle -' otherwise.
Before running any code from any of the arguments, the interpreter checks the environment variable IDLE_INIT. If its content is a string of the format '@filename', then idle loads and executes the file filename. Otherwise, idle executes the content as Idle code.
All options are handled in order, except -i. For instance, a command line like
idle -e a=7 -e print(a*a) script.idle test*.*
will first set a to 7, then print the value of a*a (which is 49), and finally run the file script.idle with a single argument of test*.*.
Before running a script, idle collects all arguments on the command line and puts them into a global table called arg. The script name is stored at index 0, the first argument after the script name goes to index 1, and so on. Any arguments before the script name (that is, the interpreter name plus the interpreter options) go to negative indices. For instance, in this call:
idle -l amod bprog.idle t1 t2
the interpreter first require( )s amod, then constructs the table arg as follows:
arg={
arg[-3] = "idle.exe",
arg[-2] = "-l",
arg[-1] = "amod",
arg[0] = "bprog.idle",
arg[1] = "t1",
arg[2] = "t2" }
and finally runs the file bprog.idle. This script can then access all the command line arguments via table arg.
In interactive mode, if you write an incomplete statement, the interpreter waits for its completion by issuing a secondary prompt. If the global Idle variable _PROMPT contains a string, then its value is used as the prompt. Similarly, if the global variable _PROMPT2 contains a string, its value is used as the secondary prompt (issued during incomplete statements). You can set these variables with the -e option:
idle -e "_PROMPT='Wot? ' _PROMPT2='More! '" -i
The Idle interpreter for GUI scripts (idlew.exe)
top of page
This version of the interpreter supports running scripts in a GUI environment. This means it doesn't open (or run in) a console window and doesn't directly support input from or output to a console (but see the functions win32.allocConsole( ) and win32.attachConsole( )). It is also possible to automatically allocate a new console window and attach the Idle stdio handles with it, such that the standard Idle output function work as expected with the newly allocated console (see the -a switch of idlew.exe).
The GUI version of the interpreter automatically require( )s the gui module, so there is no need to explicitly require( ) this (but there is no harm done if you do that).
Here is the help screen of the GUI interpreter. Most of the options have the same meaning as in the console version of the interpreter (see above).
The Idle compiler (idlec.exe)
top of page
The Idle compiler can be started with the following command line options:
usage: idlec [options] [filenames] Options are: -b name dump byte code to 'name' -x name compile to executable 'name' (default is 'out.exe') -c compress executable (but not byte code) -d include debug information -e encrypt executable (but not byte code) -l produce VM code listing -m disable token-filter-based macros -q suppress messages -w compile executable for GUI subsystem -v show version information -- stop handling options - process stdin
These options have the following meaning:
- -b: dump byte code: this compiles a .idle file (which can be a script or a module) and puts the resulting byte code into a .idol file (short for IDle Object Library) with the given name. A .idol file is simply the compiled representation of an Idle source code file. The Idle interpreter can read and execute .idol files just as well as .idle files. You have to specify a name for the .idol file.
- -x: compile: this option is the default. It takes one or more .idle files, compiles them and puts them all into an executable file. If your script requires additional modules, you may want to compile those modules into the executable. The general rule is to name any required module files before the main script (you have to give the full filenames). In this case a call to require( ) a module are satisfied by loading the precompiled chunk from the executable.
If you omit a module while compiling a script then require( ) will have to load the module at runtime, just as the interpreter does. This may or may not be what you want.
If you give no explicit -x switch, the compiler will put the executable into a file called 'out.exe'.
- -c: compress executable: this compresses the executable file (but not any files produced with -b). Compression can save around 40 to 60% of space, depending on script content. Decompression is extremely fast, so there is no performance penalty in using this switch.
- -d: include debug information: this is required if you want to use introspective functions in the debug library.
- -e: encrypt executable: the encryption used by the compiler is MD5-based and certainly not unbreakable. Encrypting an executable is more a means to ensure that the file was not tampered with than protection against attempts to reverse-engineer the source code: a determined attacker will succeed.
- -l: list VM code: this options compiles the given files and lists the VM code that is produced by the compiler. The listing can be used, among many other things, to check how many and which global and local variables are used per function.
- -m: disable macros: this disables the internal token-filter-based macro processor.
- -q: quieten the compiler: only fatal error messages will be printed.
- -w: compile for GUI subsystem: by default, the compiler produces a Win32 console application (suitable for scripts which will be run inside a console window). If this switch is given, the compiler produces a standard GUI executable (for the Windows subsystem) instead. This means, among other things, that stdin and stdout are not anymore connected to any console as a Windows subsystem executable will neither open nor access a console window; command line file redirection, however, does work as usual. (This also means you will not directly see the output of the program, if there is any.) However, a console application, if started outside a console, will allocate its own console window which may or may not be desired.
- -v: print version: shows the compiler version.
Here are a few usage examples. The following compiles test.idle and puts it into a compressed, encrypted executable called test.exe:
idlec -c -e -x test.exe test.idle
This puts the executable file into out.exe but does otherwise exactly the same as above:
idlec -c -e test.idle
All required standard Idle runtime modules are automatically included during compilation, so there is no need to explicitly include them. Third-party modules not explicitly included in a compiler command line will have to be loaded at runtime. If you want to produce a standalone executable you must specify all required modules during compilation. As an example, to compile a script that uses the (imaginary) Idle modules 'zipreader' and 'zipwriter' you would specify:
idlec -x testzip.exe zipreader.idle zipwriter.idle testzip.idle
This compiles all three .idle files and puts them into executable testzip.exe.
All Idle executables, in whatever way they were produced, need the Idle runtime DLL. They will not even start up if this library cannot be found.
Idle and system integration
top of page
Idle makes use of a few environment variables:
- IDLE_INIT: used by the Idle interpreter to execute a chunk of user-defined code before anything else is run. For security reasons, compiled Idle script do not support this environment variable.
- IDLE_PATH: This is the path used by Idle to initialise package.path. This variable, in turn, is used by require( ) to search for external Idle modules (see the detailed description of how this function locates Idle files.
- IDLE_CPATH: This is the path used by Idle to initialise package.cpath. This variable, in turn, is used by require( ) to locate dynamic libraries.
- IDLE_IPATH: This is the path used by Idle to initialise package.ipath. This variable, in turn, is used by the predefined macro __INCLUDE( ) to locate include files.
The Idle interpreter switches off the default handling for Ctrl-C while a script is executing. (The executable files produced by the Idle compiler do the same.) This means that the user cannot inadvertently interrupt a running script by pressing Ctrl-C. To terminate an Idle script (perhaps with a run-away loop) use Ctrl-Break instead. Pressing Ctrl-C may stop a script if pressed while reading standard input: this is a feature of the C runtime function used to read input.
There are two ways how an Idle script can return a numerical exit code to the operating system. One is to call os.exit( ) with the exit code; the other is to set the variable os.exitCode somewhere in the script: the value of this variable will be returned to the OS on program termination. Calling os.exit( ) is not recommended as this call terminates the process such that normal library shutdown functions are not called. This could leave some resources in an undefined state (unflushed buffers, open handles,...).
If a script does not set an explicit value for variable os.exitCode, Idle returns -1 in case there was a compile or runtime error or 0 otherwise.
The Idle runtime DLL (as well as all .EXE files) is compressed with UPX (see the UPX website). In case you want to uncompress the DLL just call UPX with the -d switch:
upx -d idle03.dll
A (very brief) tutorial on how Idle locates modules
top of page
All but the most trivial Idle scripts will call functions from one or more runtime modules. Upon program start all Idle standard runtime modules are automatically loaded and initialised (the gui module is automatically loaded by the Idle GUI interpreter (ie idlew.exe) whereas the console version has to load it by calling require( )).
Additional modules, such as the socket module, need to be loaded explicitly:
require('socket') -- load networking stuff
will locate and load the socket module (as well as its submodules like socket.http and mime). This specific module is a special case in the sense that it is automatically loaded from within the Idle runtime DLL, but if you write your own modules (or want to use third-party modules), Idle must be able to find the file(s) containing the module.
So let's, to illustrate the process, assume you call require('testmodule') to load module testmodule. By default, this searches for 'testmodule.idle' or 'testmodule.idol' and if either is found, loads it. This search is performed along the path given in environment variable IDLE_PATH or, if that is not defined, along the normal PATH.
Let's assume you have Idle installed in c:\Program Files\Idle, with all binary files in c:\Program Files\Idle\bin. Let's further assume your IDLE_PATH is set like this:
set IDLE_PATH=.\?.idle;.\?.idol;c:\Program Files\Idle\bin\?.idol;c:\Program Files\Idle\bin\?.idle
The call to require('testmodule') will first look for testmodule.idle in the current working directory. If the file is found there, it will be loaded. If not, require( ) tries to load testmodule.idol from the current working directory. If this does not succeed as well, require( ) next tries c:\Program Files\Idle\bin\testmodule.idol and finally c:\Program Files\Idle\bin\testmodule.idle.
If locating a module via IDLE_PATH fails, Idle next checks all directories in the environment variable PATH, first for testmodule.idle, then for testmodule.idol. If a matching file is found in one of these directories, it is loaded and the script is set to go. If not, Idle at long last concedes defeat and prints a tidy error message similar to this (slightly simplified) one:
test.idle:1: module 'testmodule' not found:
no field package.preload['testmodule']
no file '.\testmodule.idle'
no file '.\testmodule.idol'
no file 'c:\Program Files\Idle\bin\testmodule.idle'
no file 'c:\Program Files\Idle\bin\testmodule.idol'
no file 'c:\Program Files\Idle\bin\testmodule\init.idle'
no zio file 'testmodule.idle'
stack traceback:
[C]: in function 'require'
stdin:1: in main chunk
[C]: ?
So, if you want to ensure that Idle locates all the non-runtime modules you might require( ), just set IDLE_PATH or PATH accordingly and make sure you have put the appropriate directory (or directories) in there (setting PATH is perhaps slightly more convenient, as this already exists in most systems).
IDLE_INIT and executing and compiling a script
top of page
Let's say you just know that you will need the socket module in all your scripts (require( )ing a module even if you don't need it doesn't hurt: loading a module will at worst cost a few milliseconds during start-up). In this case you could set the environment variable IDLE_INIT as follows:
set IDLE_INIT=require('socket')
You could even include further modules you know you need all the time, simply by appending more calls ro require( ) (remember, Idle is a free-format language, so separate statements don't need to be on separate lines):
set IDLE_INIT=require('socket') require('sqlite3') require('myfancymodule')
IDLE_INIT can also be used to specify a file that should be run before any other Idle code is executed. Simply set IDLE_INIT to the filename, prefixed by an @ sign:
set [email protected]
(In case you get carried away with this feature, bear in mind that the Idle compiler and compiled Idle scripts will ignore IDLE_INIT.)
Things are slightly different if you want to compile a script to a .exe file. In this case there are two fundamentally different ways the executable can access the module(s) it needs: either they are compiled into the executable itself, making it stand-alone (however, the Idle runtime DLL is always needed). Or they could be located and loaded at runtime, much like an interpreter-run script would do. Both strategies have good and bad points.
Let's say you want to compile the following cutting-edge program:
os.sleep(150)
printf('Hello, world. I've just woken up!')
If you compile this script with a simple
idlec hello.idle
you will get an executable out.exe in which all required Idle standard runtime modules are already included. Therefore, this executable just needs the Idle runtime DLL.
However, if you want to compile a script that uses one or modules which are not part of the Idle standard runtime such as in this example:
require('testmodule')
require('anothermodule')
testmodule.testfunction(150)
anothermodule.testfunction('test')
printf('Hello, world. I've just woken up!')
then you have two choices. You could either explicitly include modules testmodule and anothermodule in the executable while compiling the .exe file. The general rule here is to first name all required module files and finally the main script file:
idlec testmodule.idle anothermodule.idle hello.idle
Or you could delay loading the modules until the executable is actually run. In this case the two require( ) calls will search and load the modules in exactly the same way as the Idle interpreter would.
The Idle distribution comes with many example scripts: just play around with them and you will quickly get a feel how all this hangs together.