# Readme/Help for PyPE (Python Programmer's Editor)

Table of Contents

- License and Contact information
- Requirements
- Installation
  - Why doesn't the Windows install work?
  - Why doesn't PyPE work on Linux?
  - Why isn't the most recent PyPE available as deb or RPM?
  - Why doesn't PyPE work on OSX?
- Command Line Options
  - --last
  - --unicode and --ansi
  - --fontsize
  - --font
  - --nothread
  - --macros
  - --standalone
  - --port
  - --use_old_parser
- PyPE features and functionality
  - What to expect when coming from other editors/IDEs
  - Encoding detection for opening files
  - Encoding detection for saving files
  - What is a "coding directive"?
  - Shells
  - Vim options
  - Using Options -> Realtime Options for syntax checking and tool updates
  - What is Sloppy Cut/Copy?
  - What is Smart Paste?
  - What is Middle Paste?
  - What do the different options in the Filter tool do?
  - How do I update the default settings for a particular document type?
  - Dictionaries and alphabets for the Spell checker
  - How does "One PyPE" work?
  - What the heck is a Trigger?
  - Find/Replace bars
  - What happens when "Smart Case" is enabled during a replace?
  - String escapes in regular expressions and multiline searches?
  - How do I use the 'Todo' list?
  - Labels / Transforms -> Insert Comment
  - What are the known issues within PyPE's parser?
    - The C/C++ parser
    - The Python parser
    - TeX/LaTeX
    - HTML/XML
    - Label Parser
    - Name/Line Expanded State
  - How do you get usable Calltips?
  - How do you get autocompletion?
  - CRLF/LF/CR line endings
  - STCStyleEditor.py
  - Expandable/collapsable/foldable code
  - Converting between tabs and spaces
- How do I program my own macros?
  - An example nontrivial macro
  - Using macros as code snippets
  - Sample Macros included with PyPE
  - Non-white background colors
- FAQ
  - How do you come up with new feature ideas?
  - What's the deal with the version numbering scheme?
  - How did PyPE come about?
- Thank Yous

# License and Contact information

http://pype.sourceforge.net http://come.to/josiah

PyPE is copyright 2003-2010 Josiah Carlson. Contributions are copyright their respective authors.

This software is licensed under the GPL (GNU General Public License) version 2 as it appears here: http://www.gnu.org/copyleft/gpl.html It is also included with this archive as gpl.txt.

The portions of STCStyleEditor.py included in StyleSetter.py, which is used to support styles, was released under the wxWindows license and is copyright (c) 2001 - 2002 Riaan Booysen.

Any software that was originally licensed as wxWindows or LGPL v2 I have relicensed as GPL v2 as is allowed under both the wxWindows and LGPL licenses.

The included stc-styles.rc.cfg was modified from the original version in order to not cause exceptions during style changes, as well as adding other language style definitions, and was originally distributed with wxPython version 2.4.1.2 for Python 2.2 .

If you do not also receive a copy of gpl.txt, with your version of this software, please inform me of the violation at either web page at the top of this document.

# Requirements

Either a machine running Python and wxPython, or a Windows machine that can run the binaries should be sufficient. Initial revisions of PyPE were developed on a PII-400 with 384 megs of ram, but modern versions have been written with more modern machines. Really, PyPE should work on any machine that can run the most recent wxPython revisions. Some portions may be slow (when using Document->Wrap Long Lines especially, which is a known issue with the scintilla text editor control in wxPython 2.8.*), but it should still be usable.

PyPE is usually only tested on my dev machine, which is currently a Windows 7 64-bit machine running 32 bit Python 2.6 with a somewhat recent wxPython 2.8. If you run into issues, please feel free to file a bug report on http://sourceforge.net/projects/pype/ .

As this document doesn't get updated all that often, please be aware that these version numbers may change during PyPE development. If those revisions of Python or wxPython are out-of-date with the bleeding edge of both pieces of software (currently Python 2.7/3.1 and wxPython 2.9.x) either I've not tested PyPE on those revisions, or I have, and I've not updated this document. PyPE has not been translated for use with Python 3.x, so it won't work. When PyPE supports Python 3.x, there will be a notification and there will be a special version.

# Installation

If you have Python 2.6+ and wxPython 2.8+, you should be able to extract the most recent PyPE-X.Y.Z-src.zip anywhere, and if your associations are set up correctly, run it by double-clicking on pype.py or pype.pyw .

If you don't have Python or wxPython installed on your system, and are using Windows, you may try to run a recent Windows binary. They are provided for your convenience, so if they don't work, please file a bug report.

## Why doesn't the Windows install work?

Depending on your platform, it may or may not work. Most problems people have is that they mistakenly extract library.zip, which they shouldn't do (and in recent PyPE binary releases may not be able to do). It could also be due to the lack of some DLL, in which case an error message should inform you of which DLL you are missing. PyPE 2.9.1 for Windows was missing msvcp71.dll file, which will be included with subsequent Windows releases.

## Why doesn't PyPE work on Linux?

If you aren't running a LTS version of Ubuntu (currently 10.4), I probably have not tested PyPE on your particular flavor of Linux. Assuming that you can download and run the wxPython demo from the "wxPython Docs and Demos" available from http://www.wxpython.org, then PyPE should also work.

There have previously been reports of PyPE segfaulting in certain Linux distributions when opening a file. This seems to be caused by icons in the file listing in the 'Documents' tab on the right (or left) side of the editor (depending on your preferences), or by icons in the notebook tabs along the top of the editor. It was due to either the platform not being able to find the icons to display, or the icons being improperly sized. You can disable these icons by starting up PyPE, going to Options->Use Icons, and making sure that it is unchecked. You should restart PyPE to make sure that the setting sticks. PyPE will be uglier, but it should work. I believe that this has been fixed in PyPE 2.4.1 and later, but this documentation persists "just in case".

If you run into other errors, please file a bug report at http://sourceforge.net/projects/pype/ .

## Why isn't the most recent PyPE available as deb or RPM?

Short answer: it's a pain in the ass.

Longer answer: I'm not the maintainer for the PyPE package in any of the Ubuntu repositories, but have recently discovered that PyPE has a newer maintainer. Whether or not the new maintainer keeps PyPE up-to-date is up to him. Personal attempts to create .debs have resulted in utter failure, which I can either blame on a personal failure to comprehend the documentation, or a failure in the documentation to impart the necessary information. Either way, you are going to have to wait for the debian/ubuntu/whatever repositories to update, or you can get the most recent PyPE from http://sourceforge.net/projects/pype and extract it wherever you desire. I'm a fan of ~/apps/PyPE, but choose what you will.

I'm not going to package any RPMs for PyPE, primarily because I'm not going to install the RPM build/install stuff into Ubuntu. Recent attempts to get bdist_wininst working in such a way that the results don't mangle Python installations have failed, and this experience leads me to believe that bdist_rpm has similar issues. Essentially, you are on your own with regards to rpm packages.

## Why doesn't PyPE work on OSX?

PyPE 2.9.1 was released with some OSX-specific optimizations to make my life using PyPE on OSX better. At the time, I had been using OSX with work, which was an experiment that lasted only 2 1/2 months, due in part to PyPE's slowness at the time on OSX. This is caused by OSX shenanigans WRT the layers that go into rendering GTK on OSX.

Since I don't have an OSX machine on which to run PyPE, I'm at the mercy of those on the system to file bug reports and test my bug fixes. PyPE works well on Linux, Windows, and within a Windows VM on OSX (where it's actually faster than in OSX native).

# Command Line Options

## --last

When PyPE is run with the '--last' command line option, PyPE will attempt to load all documents that were opened the last time you shut down PyPE. This is equivalent to starting up PyPE and using File->Open Last .

## --unicode and --ansi

If PyPE is started up with the --unicode or --ansi command line options, it will attempt to use the unicode or ansi versions of wxPython respectively. On failure, it will display to the user with a failure notice. These options have no effect on the Windows distributions of PyPE, or wherever `hasattr(sys, 'frozen')` is true.

## --fontsize

If you provide `--fontsize=12`, PyPE will change the font size for all open documents to 12. The default font size that PyPE uses is 10. If you want text to be bigger, use a number larger than 10. If you want text to be smaller, use a number smaller than 10. The line number margin will be scaled proportional to the font size specified.

## --font

If you provide `--font=Lucida-Console`, PyPE will change the font for all open documents to "Lucida Console". The default font that PyPE uses is Courier New.

## --nothread

This command line option will disable the threaded parser, which has caused problems on some platforms. This will reduce the accuracy of the tools in the "Tools" menu, due to the faster and not necessarily correct parser being used in its place.

## --macros

PyPE 2.6 has what I would consider to be a fully-functioning macro system. The Python 2.5 `--macros` command line option is now ignored because macros are enabled by default in 2.6+.

## --standalone

Providing this command line option will use the path in which the PyPE source or binary is for where PyPE's state is saved (document history, menu configuration, etc.). This will allow for 'embedded' applications.

## --port

Providing this command line option will allow you to choose the port number that PyPE uses when Options -> One PyPE is checked. The default port number is 9999.

## --use_old_parser

This uses the old parser (PyPE 2.8.8 and a few revisions prior). It is faster than the modern parser, but it's not as accurrate, nor does it provide all of the scope introspection capabilities that the new compiler.ast-based parser does.

# PyPE features and functionality

## What to expect when coming from other editors/IDEs

While PyPE has quite a few of the features that one would expect from an IDE, I do not consider PyPE to be an IDE; I consider PyPE to be an editor. The semantic difference between the two in my mind is a bit wishy-washy, so I'll not bore you with the details. In any case:

1. Hitting F5 will not run your Python, nor compile the latex, nor compile the C/C++, nor open a browser for the HTML. It will (by default) refresh the browsable source trees and other tools. You can change hotkeys, and in particular, the (new in PyPE 2.8) 'File -> Run Current File' menu item. For .py and .pyw files, 'Run Current File' will use the Python specified in the lower part of 'Options -> Shell Options' to run your Python source, capturing the output and allowing interaction. For .htm, .html, .shtm and .shtml, PyPE will try to use your system defined default web browser to open the file. For .tex files, PyPE will attempt to run pdflatex on them.
2. If PyPE seems complicated when you are first starting out, hide all of the optional features; 'Options -> Layout Options -> Show Wide Tools', 'Show Tall Tools', and '(toolbar) Hide'. Start editing. If it isn't doing what you want/expect it to, check the 'Document' menu for per-document settings or the 'Options' menu for other editor-wide options. Want to change hotkeys? Use 'Options -> Change Menus and Hotkeys' .
3. PyPE is not going to gain a debugger any time soon, if ever. I agree with many of you that debuggers can be useful, but aside from attempting to steal Idle's or some other project's remote debugger and making it work in PyPE, 1) I wouldn't know where to begin, 2) it may kill bookmark indicators, 3) I find that print statements are sufficient for me, 4) I have not had sufficient desire to make it happen.
4. PyPE is not like every other editor you have ever used. It may share some features, but it is likely just a bit different. Before you freak out and email me with, "PyPE sux, go find something else to do with your time newb! lols" spend some time looking for the feature in the menus, the various tabs, etc. You may find that your desired feature is available. Again note that if the key bindings are not to your liking, you can change them with 'Options -> Change Menus and Hotkeys' for all the menus. Macros are handled a bit differently, which you will find out by hitting the 'hotkey' button in the Macros tab.
5. PyPE has macros. These macros can record what you do with the keyboard and some menu actions, then play them back. You can also use them to programmatically edit the document you are working on, including the handling of 'code snippets'. Look at the macro help below and the samples included with PyPE (including the failure conditions).

## Encoding detection for opening files

If you are using the Unicode version of PyPE, when opening a file, PyPE will attempt to decode your file using the following encodings in order:

1. The encoding specified by the BOM, if any (PyPE writes BOMs for UTF-* encodings by default).
2. Encodings specified by "coding directives" in the first two lines of source, if any.
3. Ascii (only allows for values from 0...127)
4. Latin-1/iso-8859-1 (allows for values 0...255)

If options 1-3 above fail, then 4 will succeed, but may not necessarily display the correct content, and may cause corruption if you were to save the document.

In 2.6.3 and earlier, PyPE would try 1, 2, then 3, but not 4.

Note that PyPE does not default to assuming XML or HTML files are UTF-8 as per spec: http://www.w3.org/TR/2000/REC-xml-20001006#NT-EncodingDecl due to backwards compatability concerns with PyPE 2.6.3 and earlier. Users desiring UTF-8 decoding support should make sure that their xml/html files include a UTF-8 encoding directive or BOM at the beginning of their file, which is recommended for all xml/html anyways.

## Encoding detection for saving files

If you are using the Unicode version of PyPE, when saving a file, PyPE will attempt to encode your file using the following encodings in order:

1. Any encoding specified by the Document -> Encodings menu option (note that a specification of 'other' will be ignored, and will assume the existence of a "coding" directive.
2. Encodings specified by "coding directives" in the first two lines of source, if any.
3. Ascii (only allows for values from 0...127)
4. Latin-1/iso-8859-1 (allows for values 0...255)
5. UTF-8

If options 1-4 above fail, 5 will succeed. If the first encoding option does not succeed: say, for instance, that you have specified "other" as the Document -> Encodings option, then used the iso-8859-9 coding declaration for Turkish, but included some Arabic letters in a comment somewhere (possibly an unlikely occurrence, I don't know, but this is an example), PyPE will inform you that your intended encoding (iso-8859-9) does not match the first encoding to succeed (UTF-8), and ask you if it is ok to continue.

In 2.6.3 and earlier, PyPE would try 1, 2, 3, then 5.

## What is a "coding directive"?

If in the first two lines of your source file (all initial blank lines being ignored), the following regular expression matches something:

    [cC][oO][dD][iI][nN][gG][=:](?:["'\s]*)([-\w.]+)

... then you have a properly specified "coding directive". This regular expression was intended to match things like:

    # -*- coding: ENCODING_NAME -*-
    # -*- cOdInG: ENCODING-NAME -*-
    # vim:fileencoding=ENCODING_NAME
    <?xml version='1.0' encoding='ENCODING-NAME' ?>

... in [X]Emacs or Vim style encoding declarations for Python source, or XML-style declarations in XML or HTML source.

## Shells

PyPE includes the ability to open up Python or command shells. See the File menu. To choose which Python is used in the "New Python Shell" or "Run Selected Code", see "Options -> Shell Options".

When using "New Python shell" or the "Run Selected Code", you may notice that when you run wxPython code, any initial wx.Frame.Show() calls may not actually show the frame on Windows. To work around this, use a .Show(), followed by a .Hide(), followed by a .Show() again. This should work around the issue on Windows platforms.

When using "Run Selected Code", PyPE will try to find some open Python shell. If one is not found, PyPE will open a new Python shell using the Python specified in "Options -> Shell Options". PyPE will then send the selected code to the Python shell after reindenting it.

When using "Run Current File", PyPE will try to find a currently unused output document that was previously created. If it cannot find one, it will open a new output document and use that.

Note: as of May 2009, though shells work, there are some bugs, and seem to have become quite slow. There are some things I've been meaning to do to improve their functionality, but I've not had time (in over 2 years).

## Vim options

When opening up a file that you have never opened before, or whose history you have cleared by closing and removing it from the "Recently Open" list in the Documents tab, PyPE will scan the first and last 20 lines of the file for comments (see the Todo stuff below for what constitutes a comment), then check for :set commands. If :set commands are found, only cul, nocul, et, noet, sw, sts, ts, and their aliases (including 'inv' prefix or '!' suffix for toggles, and both '=' and ':' assignment operators for values) are used to set the preferences in the Document menu.

If there exists both sw and sts options, sw will be preferred.

## Using Options -> Realtime Options for syntax checking and tool updates

Syntax checking is always enabled for Python shells, and will highlight the first line with an error as you type (it is actually checks shortly after you stop typing), using the same indicator as defined in Options -> Shell Options.

Syntax checking for Python source files is only enabled if you have chosen a delay in the Options -> Realtime Options submenu. If your file is fewer than 200,000 bytes long, it will take max(SYNTAX_CHECK_TIME, 1)*CHOICE_IN_SECONDS, and wait that long after you have stopped using your keyboard, etc., to check the syntax, indicating the first error, if any, using the same indicator as defined in Options -> Shell Options.

Automatic source tree rebuilding for the Name and Line tools, entries for the Filter tool, Todo listing, autocomple entries, and calltips is only enabled if you have chosen a delay for update tools in the Options -> Realtime Options submenu. Otherwise you need to use Document -> Refresh (or the equivalent key binding). Similar to syntax checking above, it will take max(REFRESH_TIME, 1) *CHOICE_IN_SECONDS, and wait that long after you have stopped using your keyboard, etc., to do the automatic Document -> Refresh call.

Note that PyPE will only check syntax or rebuild the tree if the content has changed since the last time either operation was scheduled.

## What is Sloppy Cut/Copy?

When selecting multiple lines for a cut/copy operation, Sloppy Cut/Copy will select the entirety of partially selected lines. This saves the user from having to meticulously select the start and end points of multi-line selections.

## What is Smart Paste?

Smart Paste is two functionalities in one.

1. When pasting multiple lines into a currently indented region, it will reindent the pasted region such that the least indented line of the pasted region matches the current indentation level, all other indent levels being relative to the current/minimum.
2. When the cursor is in a non-indent portion of a line, and you paste, Smart Paste will automatically paste to the next line, indenting one level deeper as necessary if you had selected the start of a new block (like if, for, while, def, etc., for Python, open curly braces '{' in C, etc.).

## What is Middle Paste?

PyPE 2.9.3 and later now supports Unix-style selection + middle clicking on OSX and Linux. Windows versions of PyPE have it as an optional feature for use within PyPE only. These features are derived from Robert McMullen's post: http://goo.gl/zXsN2 .

## What do the different options in the Filter tool do?

subsequence
  will match things like `us.et` to `UserString.ExpandTabs`
score
  when subsequence is defined, will score the matches and show the best matches at the top of the list
no context
  will not provide any context in the display or search
long
  will provide a 'verbose' display and search context, like `class foo: def bar(self)`
short
  will provide a concise display and search context, like `def foo.bar(self)`
exact
  will find entries that include exactly what you typed in.
any
  will find entries that include any of the 'words' you provide.
all
  will find the entries that include all of the 'words' you provide

Given the following three definitions and the `no context` option without subsequence searching:

    def abc(ghi, jkl)
    def jkl(mno, pqr)
    def stu(vwx, yz)

...the following searches are true:

    exact 'def abc' -> #1
    any 'def abc' -> #1, #2, #3
    all 'def abc' -> #1

    exact 'abc ghi' -> Nothing
    any 'abc ghi' -> #1
    all 'abc ghi' -> #1

    exact 'jkl stu' -> Nothing
    any 'jkl stu' -> #1, #2, #3
    all 'jkl stu' -> Nothing

With the new parser introduced in PyPE 2.9, line count information should be fairly precise.

## How do I update the default settings for a particular document type?

1. Close all open documents of the particular type whose default settings you want to update.
2. Create or open a document of the specific document type that you want to change the settings of.
3. Adjust all of the settings in the "Document" menu to those settings that you want to be the default when you open up that particular kind of document.
4. Use "Options -> Save Settings" and choose the particular language whose settings you would like to save.
5. If in the future, a particular document of that type does not have the proper settings, use "Options -> Load Settings" to load the defaults for that specific language.

In PyPE 2.6.3 and later, whenever a document shares the default settings for its file type and is closed, those settings aren't explicitly saved, under the assumption that you would prefer to have it use the default settings directly. If you are going to change the default settings for all documents of a specific type, follow the above 5 steps.

## Dictionaries and alphabets for the Spell checker

You can create/delete custom dictionaries via the +/- buttons right next to the "Custom Dictionaries:" section. You can add words to these custom dictionaries by "Check"ing your document for misspellings, checking all of the words you want to add, clicking "+ ./", then choosing the custom dictionary you want the words added to.

If you want to use a large primary dictionary, create a 'dictionary.txt' file that is utf-8 encoded, and place it into the same path that PyPE is. This will be far faster for startup, shutdown, and creating the list than manually adding all of the words to custom dictionaries. Fairly reasonable word lists for english (British, Canadian, or American) are available at Kevin's Word list page: http://wordlist.sourceforge.net/ Words should be separated by any standard whitespace character (spaces, tabs, line endings, etc.).

If you want to customize the alphabet that PyPE uses for suggesting spelling, you can create an 'alphabet.txt' file that is utf-8 encoded, where alphabet characters separated by commas ',', and place it into the same path that PyPE is.

Please note that the spell checker is very simple. After discovering "words", which are contiguous sequences of letters, suggestions are created by removing single letters, inserting single letters, and swapping pairs of letters internally. It then checks these suggestions against the user-supplied dictionaries, and any that match become suggestions.

## How does "One PyPE" work?

If "One PyPE" is selected, it will remove the file named 'nosocket' from the path in which PyPE is running from (if it exists), and start a listening socket on 127.0.0.1:9999 . If "One PyPE" is deselected, it will create a file called 'nosocket' in the path from which PyPE is running, and close the listening socket (if one was listening).

Any new PyPE instances which attempt to open will check for the existence of the nosocket file. If it does not find that file, it will attempt to create a new listening socket on 127.0.0.1:9999 . If the socket creation fails, it will attempt to connect to 127.0.0.1:9999 and send the documents provided on the command-line to the other PyPE instance. If it found the file, or if it was able to create the socket, then a new instance of PyPE will be created, and will use the preferences-defined "One PyPE" (preventing certain issues involving a read-only path which PyPE is on, or a read-only nosocket file).

If you want to prevent new instances of PyPE from ever creating or using sockets, create a file called 'nosocket' and make it read-only to PyPE.

## What the heck is a Trigger?

Let us say that you writing a web page from scratch. Let us also say that typing in everything has gotten a bit tiresome, so you want to offer yourself a few macro-like expansions, like 'img' -> '<img src="">'.

1. Go to: Document->Set Triggers.
2. Click on 'New Trigger'.
3. In the 'input' column of the new trigger, type in `img`
4. In the 'output' column, type in `<img src="%C">`

In the future, if you type in `img` and use Transforms->Perform Trigger, it will expand itself to `<img src="">` with your cursor between the two double quotes.

What other nifty things are possible? How about automatic curly and square brace matching with [, [%C] and {, {%C}? Note that triggers with a single character in the 'enter' column are automatically done as you type, but triggers with multiple characters in the 'input' column require using Transforms->Perform Trigger (or its equivalent hotkey if you have assigned one via Options -> Change Menus and Hotkeys).

As described, there is a `%C` directive that defines where the cursor will end up. There is also a `%L` directive that inserts a line break with autoindentation. The semantics for string escapes are the same as in the Find/Replace bar, and a non-indenting line break can be inserted with the standard `\n`.

## Find/Replace bars

If you have ' or " as the first character in a find or find/replace entry, and what you entered is a proper string declaration in Python, PyPE will use the compiler module to parse and discover the the string. For example, to discover LF characters, use `"\n"`, including quotes.

## What happens when "Smart Case" is enabled during a replace?

If the found string is all upper or lower case, it will be replaced by a string that is also all upper or lower case.

Else if the length of the found string is the same length as the replacement string, you can replace one string for another, preserving capitalization.

For example...

    def handleFoo(foo, arg2):
        tfOO = fcn(foo)
        tFOO2 = fcn2(tfOO)
        return fcn3(tfOO, tFOO2, foo)

...becomes...

    def handleGoo(goo, arg2):
        tgOO = fcn(goo)
        tGOO2 = fcn2(tgOO)
        return fcn3(tgOO, tGOO2, goo)

...by enabling "Smart Case", and putting 'foo' and 'goo' in the find/replace boxes.

Otherwise if the first letter of the found string is upper or lowercase, then its replacement will have the first letter be upper or lowercase respectively.

## String escapes in regular expressions and multiline searches?

When using the 'Search' tab, you can use standard Python strings with escapes and quote marks just like when you use the find/replace bars with one minor difference; all searched data is normalized to have `\n` line endings regardless of the input. This means that if you want to find a colon followed by a line ending followed by a space, you would use `":\n "`, including quotes.

If you include line endings in your search string, then multiline searching will be automatically enabled during the search (but the box will remain checked or unchecked).

## How do I use the 'Todo' list?

On a line by itself (any amount of leading spaces), place something that matches the following regular expression: `([a-zA-Z0-9 ]+):(.*)` and is immediately proceeded with a language-specific single-line comment (`#`, `//`, `%`, or `<!--`).

The first group (after a .strip().lower() translation) will become category in the 'Category' column, the second group (after a .strip()) becomes the todo in the 'Todo' column, and the number of exclamation points will become the number in the '!' column.

PyPE should also toss all entries with a 'Category' that is also a keyword (keyword.kwlist), or one of the following: http, ftp, mailto, news, gopher, and telnet.

The following lines are all valid todos

    # todo: fix the code below
            #todo:fix the code below!
        #        TODo: fix the code below
    #bug:I am a big ugly bug...no, I really am, but I'm also a todo
    # this thing can even have spaces: but it cannot have punctuation!

    #I am not a valid todo...: because there is punctuation on the left

In PyPE 2.6.5 and later, for Python, C/C++, and TeX files, PyPE supports the use of `#>` (or equivalents for non-XML/HTML languages) as a "strict" todo, with the option to only recognize these "strict" todos.

## Labels / Transforms -> Insert Comment

When you use Transforms -> Insert Comment, you create a comment of the form (for example in Python):

    #--------------------- comment ---------------------

With your comment centered, and the comment filling up the number of columns defined via Document -> Set Long Line Column. Such comments will show up as 'labels' within the Name, Line, and Filter tools as:

    -- comment --

This works similarly to SPE's display of such labels, but PyPE trims extraneous dashes and spaces from either end, inserting a single space and a double dash around the comment (for consistency and readability).

## What are the known issues within PyPE's parser?

### The C/C++ parser

PyPE 2.6.1 and later added a C/C++ parser that uses a combination of regular expressions and some post-processing to extract function definition information. Note that it can handle things like the following and their variations:

    int ** foo(char* arg1, int larg1) \{ ...

    str1 myClass :: operator[] (indices, count)
    int* indices;
    int count;
    \{ ...

Generally speaking, it searches for all matches of the following regular expressions for function-like examples of `#define` and functions respectively:

    (#ys+i\(i(?:,s*i)*\))
    (?:(cs*\([^\)]*\))[^{;\)]*[;{])

Where the following replacements are made to the regular expressions prior to matching:

    c -> (?:i|operator[^\w]+)
    i -> (?:[a-zA-Z_]\w*)
    s -> [ \t]
    y -> (?:[dD][eE][fF][iI][nN][eE])

The function-like macros are returned unchanged, while the possible function matches have various other tests performed on them and everything on the same line as the potential function definition.

Note that the parser doesn't recognize struct definitions, data members of classes, class hierarchies, functions with default values, etc., but it should generally be sufficient for most navigation and/or file-specific autocomplete and calltips.

### The Python parser

For Python source files, if given a syntactically correct Python source file, the Python parser should work without issue (as long as --nothread is not provided), though it may not be quite as fast as desired (where fast is < .1 seconds). Recent versions of PyPE have a much faster "slow" parser than previous versions, but it is still limited to syntactically correct source files.

If not given a syntactically correct Python source file (or if --nothread was provided as a command line option), the parser splits the file into lines, performing a check to see if there is a function, class, or comment on that line, then saves the hierarchy information based on the level of indentation and what came before it. This can be inaccurate, as it will mistakenly believe that the below function 'enumerate' is a method of MyException.

    class MyException(Exception):
        pass
    try:
        enumerate
    except:
        def enumerate(inp):
            return zip(range(len(inp)), inp)

It also doesn't know anything about multi-line strings, so the definition nada in the following lines would be seen as a function, and not part of a string.

    '''
    this used to be a function
    def nada(inp):
        return None
    '''

This parser will not pull out doc strings or handle multi-line function definitions properly (which can be difficult if not impossible when provided with a bad source file).

### TeX/LaTeX

In TeX/LaTeX, PyPE extracts \(sub)*section and \label headings, todo items, and labels (defined below).

### HTML/XML

PyPE only extracts todo items and labels (defined below).

### Label Parser

Knowing where to insert a label (in the trees) is tricky work, and we can only generally choose the right place to insert labels in one of the following two cases:

    def foo():
        #-- label 1 --
        ...

    #--label 2--

Relying on indentation for these is not generally reliable, so we place it in the context of the scope of the following function/class/whatever definition. The following source:

    class foo:
        def bar(self):
            #-- label 1 --
            def goo():
                #-- label 2 --
                ...
        #-- label 3 --
        def baz(self):
            #-- label 4 --
            ...
        #-- label 5 --

Will have a general tree layout of:

    class foo:
        def bar():
            -- label 1 --
            def goo():
        -- label 2 --
        -- label 3 --
        def baz():
    -- label 4 --
    -- label 5 --

Using a 'previous definition' semantic, we get a layout of:

    class foo:
        def bar():
            -- label 1 --
            def goo():
                -- label 2 --
                -- label 3 --
        def baz():
            -- label 4 --
            -- label 5 --

Which is different, but not substantially better, and may hide labels. It is better to show too many labels in a particular context than too few.

### Name/Line Expanded State

PyPE will only be able to remember those items that were expanded, selected or first visible (to keep the scrollbar consistant) if the names hadn't been changed. Say that you had an item named `class foo:` that was expanded prior to using Document -> Refresh. If you renamed it to `class foo_bar:`, then PyPE wouldn't remember that it was expanded in the browsable source tree.

Also, if you have two classes with the same name like the following:

    if CONDITION:
        class foo:
            def bar(self):
                ...
    else:
        class foo:
            def bar(self):
                ...

And one was expanded in the Name (or Line) tool, then both will be expanded in the Name (or Line) tool.

## How do you get usable Calltips?

Hit F5. This will also rebuild the browsable source tree, autocomplete listing, filter, and todo list.

## How do you get autocompletion?

Easy. In the 'Document' menu, there is an entry for 'Show autocomplete'. Make sure there is a check by it, and you are set. If you want to get a new or updated listing of functions, hit the F5 key on your keyboard.

## CRLF/LF/CR line endings

PyPE will attempt to figure out what kind of file was opened, it does this by counting the number of different kinds of line endings. Which ever line ending appears the most in an open file will set the line ending support for viewing and editing in the window. Also, any new lines will have that line ending. New files will have the same line endings as the host operating system.

Additionally, copying from an open document will not change the line-endings. Future versions of PyPE may support the automatic translation of text during copying and pasting to/from the host operating system's native line endings.

Converting between line endings is a menu item that is available in the 'Document' menu.

## STCStyleEditor.py

As I didn't write this, I can offer basically no support for it. It seems to work to edit python colorings, and if you edit some of the last 30 or so lines of it, you can actually use the editor to edit some of the other styles that are included.

If it doesn't work for you, I suggest you revert to the copy of the editor and stc-styles.rc.cfg that is included with the distribution of PyPE you received. As it is a known-good version, use it.

## Expandable/collapsable/foldable code

Since the beginning, there have been expandable and collapsable scopes thanks to wxStyledTextCtrl. How to use them... Given the below...

    - class nada:
    -     def funct(self):
    -         if 1:
    |             #do something
    |             pass

Shift-clicking the '-' next to the class does this...

    - class nada:
    +     def funct(self):

Or really, it's like ctrl-clicking on each of the functions declared in the scope of the definition. Shift-clicking on the '-' a second time does nothing. Shift-clicking on a '+' expands that item completely.

Control-clicking on a '+' or '-' collapses or expands the entirety of the scopes contained within.

I don't know about you, but I'm a BIG fan of shift-clicking classes. Yeah. Play around with them, you may like it.

## Converting between tabs and spaces

So, you got tabs and you want spaces, or you have spaces and want to make them tabs. As it is not a menu option, you're probably wondering "how in the hell am I going to do this". Well, if you read the above stuff about string escapes in the find/replace bar, it would be trivial. Both should INCLUDE the quotation marks. To convert from tabs to 8 spaces per tab; replace `"\t"` with `" "` To convert from 8 spaces to one tab; replace `" "` with `"\t"`

Note that you don't need to use the double quotes for the spaces, but it allowed me to be explicit in this documentation.

Alternatively, this is available via the "Transforms" menu.

# How do I program my own macros?

Users of PyPE 2.5.1 (a test release) and later will have the ability to record, edit, playback, and delete macros. Most keyboard related tasks are recorded (typing, keyboard movement, selection, cut, copy, paste, etc.), as are all items in the Transforms menu; including automatic and manual triggers.

Any macro without any action performed will not be recorded. That is, if you hit "Start Recording" and do nothing other than hit "Stop Recording", a macro will not be created. If you would like to create an initially empty macro, you can use "Empty Macro" and it will get everything all set up for you.

Before you execute your macro, I encourage you to save all currently open documents. While I haven't experienced any recent crashes or segfaults while using macros, I may not be able to replicate your particular crash condition even if given the macro source, so may not be able to fix your problem. Be careful!

Let us assume that you have created an initially empty macro with the "Empty Macro" button, whose contents are something like the following:

    creation_date = 'Wed Jul 12 21:35:34 2006'
    name = 'macro: Wed Jul 12 21:35:34 2006'
    hotkeydisplay = ""
    hotkeyaccept = ""

    def macro(self):
        pass

`creation_date`
  is merely for reference purposes
`name`
  is the name you will see in the macro list. If this value is missing, you will see the file name instead.
`hotkeydisplay`
  if you have created a hotkey for this macro, this represents how the hotkey would be displayed to PyPE. To get usable values for `hotkeydisplay`, use the 'Create Hotkey' button.
`hotkeyaccept`
  if you have created a hotkey for this macro, this represents the actual underlying keyboard keypresses necessary to make the macro run. To get usable values for `hotkeyaccept`, use the 'Create Hotkey' button.
`def macro(self):`
  is the initial definition of the macro. You can have any number of helper functions, extra data, etc., but the macro itself must be named `macro`, and must take at least one argument, the first of which being the `wxStyledTextCtrl` instance that contains the current document. You can also import any module that is available (which may be limited on systems using the Windows binary).

The `self` parameter will actually be my own custom subclass of the `StyledTextCtrl`. You will never receive a shell or interpreter, and you will not be able to execute macros on shells or interpreters.

Generally speaking, the `wxStyledTextCtrl` subclass has everything that the normal control subclass has, with a few caveats.

1. `self.GetText()` and `self.SetText()` will return and set the content of the document, paying attention to encodings as necessary. That is, if you perform `y = self.GetText()` inside a macro on a document including unicode characters, or a document defining one of the standard Python document encoding methods, you will receive the encoded version of your document. Strictly ASCII documents or those without any encodings will produce the document as-is.

   If you would like to acquire the contents of the file as-is, unicode on unicode platforms, etc.:

       import wx.stc

       def macro(self):
           content = wx.stc.StyledTextCtrl.GetText(self)

2. `self.lines` is a special property that gives you a line-based view of the current document.:

       line = self.lines[i]        # will return line "i" including whitespace
       lines = self.lines[i:j]     # will return lines i...j-1, using standard Python slice semantics
       bad = self.lines[i:j:-1]    # will raise an exception (only steps == 1 are acceptable)

       self.lines[0] = 'hello world\n' # will set the first line to be "hello world"
       self.lines[0] = 'hello world '  # will set the first line to be "hello world ",
                                       # and the next line will become the tail end of the first line

       del self.lines[i] # same as self.lines[i] = ''

       #other special properties of self.lines:
       self.lines.curline          # manipulation of the line the cursor is on
       self.lines.curlinei         # manipulation of the index where the cursor is
       self.lines.curlinep         # manipulation of the column in the line where the cursor is
       self.lines.selectedlines    # manipulation of the lines where the selection exists
       self.lines.selectedlinesi   # manipulation of the indices where the selection exists
       self.lines.targetlines      # manipulation of the lines where the target exists
       self.lines.targetlinesi     # manipulation of the indices where the target exists
                                   # the target is like an invisible selection

       #to force the selection of all of all lines where a selection currently exists:
       self.lines.selectedlinesi = self.lines.selectedlinesi

       #to iterate over the indices of all selected lines:
       for i in xrange(*self.lines.selectedlinesi):
           ...

       #etcetera.

3. `self.InterpretTrigger(text)` will interpret the text you provide as it would interpret a trigger, with a small change. That is,:

       self.InterpretTrigger('def foo(%C):\npass')

   will produce the following, with the cursor where the `@` is, without the `@` sign.:

       def foo(@):
           pass

   If you want your `'\n'` line endings to not include auto-indenting (as is the default for normal triggers), use `self.InterpretTrigger(text, 1)`.

4. `self._autoindent(0)` will perform the equivalent of `self.InterpretTrigger('\n')`.

## An example nontrivial macro

When I was writing macro support, I would have found macros to be quite convenient for developing macros. What do I mean? Let us say that I wanted to turn a line that read (from main_window_callback.c in the gPHPedit sources):

    case (2316) : gtk_scintilla_document_start(GTK_SCINTILLA(main_window.current_editor->scintilla)); break;

Into a line that read:

    2316: 'DocumentStart',

As I ended up doing by hand. Well, I could write the following macro, select those lines I wanted to update, and execute the macro.:

    def macro(self):
        lines = self.lines
        newlines = []
        for i in xrange(*lines.selectedlinesi):
            line = lines[i]
            pieces = line.split()
            num = pieces[1].strip('()')
            name = pieces[3]
            name = name.split('(', 1)[0].title()
            name = ''.join(name.split('_')[2:])
            newlines.append("    %s: '%s',"%(num, name))
        lines.selectedlines = newlines

Presumably one would want to include error handling in your nontrivial macros, but that shouldn't be terribly difficult.

## Using macros as code snippets

1. Create a macro.
2. Paste the content of your snippet into a global variable in the macro and call it something like `snippet`.
3. Use `self.InterpretTrigger(snippet)`.

That is, let us say that you wanted a snippet that inserted the following content:

    def foo(bar):
        pass

You would create the following macro:

    name = 'Code Snippet foo()'

    snippet = '''
    def foo(bar):
        pass
    '''

    def macro(self):
        self.InterpretTrigger(snippet, 1)

## Sample Macros included with PyPE

PyPE includes a handful of sample macros to give you some idea of what works and what doesn't. The most important ones you should look at are the various Timeout macros. They will show you what things will and won't stop after the 5 second timeout. The timeout conditions are there to try to prevent you from trying to kill PyPE because it stopped responding. The general rule of thumb: don't perform any system calls that could take a long time to finish.

## Non-white background colors

In PyPE 2.8.6, the stylesetter now has support for non-white background colors. To set a non-white background color, change the 'backcol' value in the proper common.defs.* in your 'stc-styles.rc.cfg'.

# FAQ

## How do you come up with new feature ideas?

Every once and a while, I'll be editing with PyPE, and I'll say, "hey, it would be neat if I could do X with PyPE". This is rare, though it has produced things like the dragable document list, spell check, customizable menu hotkey bindings, open module, "One PyPE", etc.

More often than not, I will be surfing the net, and someone will rant and rave about their super ultra mega favorite editor X, and how it has so many features that are so great that no other editor has. Out of curiosity, I'll usually go to the specific site, look at the editor, the features it offers, and consider if I would want PyPE to have such features, what changes would be necessary, and what it would take to make them happen. This has produced things like workspaces, shells, find/replace bars (idea from Firefox), triggers (and most everything else in the Transforms menu), the name and line oriented browsable source trees, etc.

Occasionally, some user of PyPE will contact me, perhaps report a bug, or somesuch, and eventually either suggest features or offer up patches. While I had written the original Search tab, the current Search tab and the table display of results were submitted almost complete. Suggestions have resulted in the addition of Start/End selection, bookmarks, the line-based abstraction for macros, macros themselves, tools whose positions can be switched, title options, the optional toolbar, caret tracking and width options, find/replace bar history, the actual find/replace bar keybindings and what they do based on context, the embedded HTML help, the Find Definition/filter tool, etc.

Astute observers will note that I have not really come up with anything terribly original myself. However, through observing other editors and IDEs, and receiving great suggestions from users, I think that PyPE has managed to acquire some very useful features. Generally, I have written PyPE primarily for myself, so if tools have a particular aesthetic or design, it's so that look and work according to how I think they should (the exception being how document preferences are handled, I really need to change that design). I hope that others find PyPE as natural to use as I do, but if not, then I welcome your feedback.

## What's the deal with the version numbering scheme?

Early in development, PyPE raised version numbers very quickly. From 1.0 to 1.5, not much more than 2 months passed. In that time, most of the major initial architectural changes that were to happen, happened. This is not the reason for the version number change. Really it was so that the MAJOR versions could have their own point release (1.0 being the first), and minor bugfixes on the point releases would get a minor release number (like 1.0.1).

Then, at around PyPE 1.4.2, I had this spiffy idea. What if I were to release a series of PyPE versions with the same version numbers as classic Doom? I remembered updating to 1.1, then to 1.2a, etc. My favorite was 1.666. Ah hah! PyPE 1.6.6.6, the best version of PyPE ever.

I decided that I would slow version number advancement, if only so that people didn't get sick of new releases of PyPE being numbered so much higher when there were minimal actual changes. Then the more I thought about it, the more I realized that it doesn't matter at all, I mean, Emacs is on version 20+. *shrug*

When PyPE 1.9.3 came out, I had a few other ideas for what I wanted to happen, but since major changes to the underlying architecture were required, it really should get a major number bump to 2.0. After spending 3 months not working on PyPE May-July 2004, I got some time to muck around with it here and there. After another few months of trying to rebuild it to only require a single STC (with multiple document pointers, etc.) I realized that I'd have to rebuild too much of PyPE to be able to get 2.0 out the door by 2010. So I started modifying 1.9.3. All in all, around 85% of what I wanted made it into PyPE 2.0, the rest was either architectural (ick), or questionable as to whether or not anyone would even want to use the feature (even me).

## How did PyPE come about?

The beginnings of PyPE were written from 10:30PM on the 2nd of July through 10:30PM on the 3rd of July, 2003. Additional features were put together on the 4th of July along with some bug fixing and more testing for version 1.0. Truthfully, I've been using it to edit itself since the morning of the 3rd of July, and believe it is pretty much feature-complete (in terms of standard Python source editing). There are a few more things I think it would be nice to have, and they will be added in good time (if I have it).

One thing you should never expect is for PyPE to become an IDE. Don't expect a UML diagram. Don't expect a debugger. Don't expect debugging support (what, print statements not good enough for you?)

On the most part, this piece of software should work exactly the way you expect it to...or at least the way I expect it to. That is the way I wrote it. As a result, you don't get much help in using it (mostly because I am lazy). There was a discussion of a PyPE wiki a long time ago, but that will likely never happen (I've lost contact with the people who initially put forward the wiki idea, and I have no interest in starting or maintaining one).

The majority of the things that this editor can do are in the menus. Hotkeys for things that have them are listed next to their menu items, and you can both rename menu items and change their hotkeys via Options->Change Menus and Hotkeys.

# Thank Yous

Certainly there are some people I should thank, because without them, the piece of software you are using right now, just wouldn't be possible.

Guido van Rossum - without Guido, not only would I not have Python, I also wouldn't have had some of the great inspiration that IDLE has offered. IDLE is a wonderful editor, has some excellent ideas in terms of functionality, but unfortunately does not offer the extended functionality I want, and it hurts my brain to use tk, so I cannot add it myself.

Robin Dunn - without Robin Dunn spending countless hours wrapping and building Pythonic APIs, I'm not sure that PyPE could have come to be. Most of the other GUI libraries I was looking at the time for doing PyPE development have fallen behind, and/or stopped being maintained. Robin deserves more thanks than I could possibly express in this blurb.

The people writing wxWidgets (previously named wxWindows) - without you, this also would not have been possible. You have made the most self-consistent GUI libraries that I have ever used, made them easy to use, and offer them on every platform that I would ever want or need. You rock.

Neil Hodgson and others who work on Scintilla. As wx.StyledTextCtrl is a binding for scintilla in wxWidgets, which then has bindings for wxPython, basically ALL the REAL functionality of the editor you are now using is the result of Scintilla. The additional things like tabbed editing, hotkeys, etc., they are mere surface decorations in comparison to what it would take to write everything required for a text editor from scratch. Gah, an editor widget that just works? Who would have figured?

To everyone who I have already thanked: thank you for making PyPE an almost trivial task. It would have been impossible to go so far so fast by hand in any other language using any other GUI toolkit or bindings.