Contributing

Development process

Code changes should be made in the following way:

• Create a new branch.
• Add code and unit tests. All code should be covered with tests.
• Run tests locally.
• Make a pull request and make sure that travis build succeeds.

Local copy

Getting local copy:

$git clone https://github.com/Ahhhhmed/homotopy.git

Running tests (run from the project root):

$python -m unittest

Internals

The code is separated in several components:

• Preprocessor
• Parser
• Syntax tree
• Snippet provider
• CodeGenerator
• Application frontend

Preprocessor

Before parsing and creating syntax tree some prepossessing is done to enable some feathers. They are described in following sections.

Decorators

Consider following example:

1. for#int$i%n
2. for[[sup]]

And following snippet definition:

{"name": "sup","language": "C++","snippet": "#int$i%n"}

This pattern occurs often and it is useful to name it and use it by name (‘sub’ stands for ‘standard up’). More realistic examples are design pattern implementation (singleton for example). Another advantage of this approach is that multiple decorators can be combined to make a more complex construction.

Preprocessor detects decorators that are defined in double square brackets and expands them using snippet provider.

Cursor marker

After expanding a snippet user should have the cursor at a convenient location. The following rule is followed.

Writing some text after expanding the snippet should have the same effect as writing that text and expanding the snippet afterwords.

To enable this preprocessor appends &[{cursor_marker}] to the snippet text so a plugin can put cursor at the marker location.

Usage

class homotopy.preprocessor.Preprocessor(snippet_provider)

Prepossess snippet text to enable extra feathers.

expand_decorators(snippet_text)

Expand decorators to enable concise writing of common patterns.

Parameters:snippet_text – Snippet text Returns:Expanded snippet text

static put_cursor_marker(snippet_text)

Put cursor marker witch should be used by editor plugins for better experience.

Parameters:snippet_text – Snippet text Returns:Snippet text with marker at the end

from homotopy.preprocessor import Preprocessor
from homotopy.snippet_provider import SnippetProvider

preprocessor = Preprocessor(SnippetProvider('python', ['folder1', 'folder2']))
snippet = "for"
expanded_snippet = preprocessor.expand_decorators(snippet)
expanded_snippet_with_cursor = preprocessor.put_cursor_marker(expanded_snippet)

Parser

Responsible for converting snippet string to a syntax tree instance.

Parameters

Basic functionality is adding parameters to a snippet.

snippet#parameter1$parameter2

This should become:

          $
         / \
        /   \
       #    parameter2
      / \
     /   \
snippet  parameter1

Following characters are used to indicate parameters:

{'!', '@', '#', '$', '%', ':', '~'}

Inside snippets

Snippets can have other snippets insight them (like body of a for loop for example). Snippets are separated by special characters that determine their relations.

• > move to the inside of a snippet
• < move to another snippet on the level above
• & move to another snippet on the same level

Example:

for>if>if<if&if

This should be translated to:

         >
        / \
       /   \
      >    if
     / \
    /   \
   >    if
  / \
 /   \
for   >
     / \
    /   \
   if   if

Character > is used to donate the inside of a snippet in snippet definitions. This is why all occurrences of < and & are translated to >.

Implicit snippet

Consider following example:

for>if<while

The while is not a part of for snippet. Parser creates an implicit block snippet at the beginning. block snippet is implemented as a list of sub-snippets in new lines.

Escape sequence

Homotopy uses escape sequence to enable operator usage in snippets. Character after '\' will always be a part of snippet and not recognised as an operator.

Usage

class homotopy.parser.Parser

Class for parsing a string to produce a syntax tree.

static parse(snippet_text)

Parsing a string to produce syntax tree.

Parameters:snippet_text – Text to parse Returns:syntax tree instance corresponding to given text

from homotopy.parser import Parser

parser = Parser()
snippet = "for"
syntax_tree = parser.parse(snippet)

Syntax tree

Tree structure of a snippet.

class homotopy.syntax_tree.CompositeSnippet(left, operation, right)

CompositeSnippet compose two snippets with the operand

accept(visitor)

Accepts a visitor

Parameters:visitor – Visitor instance Returns:Visitor result

class homotopy.syntax_tree.SimpleSnippet(value)

BasicSnippet can be directly compiled.

accept(visitor)

Accepts a visitor

Parameters:visitor – Visitor instance Returns:Visitor result

class homotopy.syntax_tree.Snippet

Base class for snippet syntax tree.

accept(visitor)

Accepts a visitor

Parameters:visitor – Visitor instance Returns:Visitor result

class homotopy.syntax_tree.SnippetVisitor

Base class for visitors working on syntax tree.

visit_composite_snippet(composite_snippet)

Base visit logic for composite snippets. Process right and left subtrees recursively.

Parameters:composite_snippet – CompositeSnippet instance Returns:None

visit_simple_snippet(simple_snippet)

Base visit logic for simple snippets. Does nothing.

Parameters:simple_snippet – SimpleSnippet instance Returns:None

Usage

from homotopy.syntax_tree import SimpleSnippet, CompositeSnippet

simple_snippet = SimpleSnippet("for")
composite_snippet = CompositeSnippet(simple_snippet, '>', SimpleSnippet('code'))

Snippet provider

Provides snippets definitions from a database of snippets.

Snippet definition files

Snippets definition are writen in json files as shown is the following example.

[
{"name": "for","language": "C++","snippet": "for(###){$$$}"},
{"name": "if","language": ["C++", "java"],"snippet": "if(###){$$$}"}
]

Note that language can be a string or a list of strings. Use all for snippets that should always be included. Language can be excluded by prefixing it with ~ (for example ~c++).

Snippet provider reads all the files containing snippets. It searches all json files contained in the given list of folders files. The list of folders is specified in the path variable (similar to os.path).

Usage

class homotopy.snippet_provider.SnippetProvider(language, path)

Class for translating simple snippets into code. Uses a database provided in json files located in the path variable.

__init__(language, path)

Initialize snippet provider instance.

Parameters:

• language – language to compile to
• path – list of directories to search for json files containing snippets

__getitem__(item)

Expand single snippet.

Parameters:item – snippet Returns:snippet expansion

from homotopy.snippet_provider import SnippetProvider

provider = SnippetProvider("C++", ["folder1", "folder2"])
snippet = "for"
snippetExpansion = provider[snippet]

CodeGenerator

CodeGenerator is responsible for turning a syntax tree into output code. It uses snippet provider for simple snippets. General rules used by code generator will be discussed in this section.

Simple substitution

Consider the following snippet definition.

{"name": "if","language": "C++","snippet": "if(###){$$$}"}

Snippet if#true$i=3; is expanded in the following way:

• if becomes if(###){$$$} from the definition.
• ### gets replaced by true to get if(true){$$$}.
• $$$ gets replaced by i=3; to get the final output if(true){i=3;}.

The value of an operator gets replace by the value provided in the snippet. This is done for every operator to get the final result. Note that there are 3 characters in snippet definition and only one in the snippet. The reason for this is that special characters used by homotopy are also used by other programming languages. For example, $ is a part of a variable name in php.

Definition expansion

Consider a snippet for a function call. Writing fun!foo#a#b#c should return foo(a,b,c). To write a single snippet definition for all functions would mean supporting variable number of parameters.

This is possible using snippet definitions inside snippets.

[
{"name": "fun","language": "python","snippet": "!!!({{params}})"},
{"name": "params","language": "python","snippet": "###{{opt_params}}"},
{"name": "opt_params","language": "python","snippet": ", ###{{opt_params}}"}
]

Snippet fun!foo#a#b is expanded in the following way:

• fun becomes !!!({{params}}) from the definition.
• !!! gets replaced by foo to get foo({{params}}).
• ### does not exist in foo({{params}}) so {{params}} get expanded to ###{{opt_params}}.
• ### gets replaced by a; to get foo(a{{opt_params}})
• ### does not exist in foo(a{{opt_params}}) so {{opt_params}} get expanded to , ###{{opt_params}}.
• ### gets replaced by b; to get foo(a, b{{opt_params}}).
• {{opt_params}} gets removed from final result to get foo(a,b).

Expansion is done in case simple substitution can’t be done. This enables recursive constructs as shown in the example above.

Note that there might be multiple sub-snippets inside a single snippet. In that case the first one containing required operator in its definition gets expanded. Other sub-snippets do not get expanded.

Outer parameters

Accessing outer parameters can be done in the following way:

[
{"name": "constructor","language": "java","snippet": "public {{?###}}(){}"}
]

The snippet above would create a public empty constructor. {{?###}} binds to the same value as {{?###}} from the snippet above the current one.

Usage

class homotopy.code_generator.CodeGenerator(snippet_provider, indent_manager)

Compiler for snippets. Turns syntax tree into text.

generate_code(snippet)

Generate code for a snippet. Visit and then perform a clean.

Parameters:snippet – Snippet Returns:Text of compiled snippet

from homotopy.code_generator import CodeGenerator
from homotopy.parser import Parser
from homotopy.snippet_provider import SnippetProvider
from homotopy.util import IndentManager

snippet_provider = SnippetProvider('python', ['folder1', 'folder2'])
indent_manager = IndentManager(snippet_provider)

code_generator = CodeGenerator(snippet_provider, indent_manager)
parser = Parser()

snippet = "for>code"
syntax_tree = parser.parse(snippet)

compiled_snippet = code_generator.generate_code(syntax_tree)

Application frontend

Homotopy class

Homotopy class scarves as a facade for the whole tool. It should be the eatery point for snippet compilation. It can be configured to include additional paths to user defined snippet libraries.

Example:

from homotopy import Homotopy

cpp_snippets = Homotopy("c++")
print(cpp_snippets.generate_code('int n=5;&for#int$i%n>printf("hello");'))

outputs:

int n=5;
for(int i=0; i<n; i++){
    printf("hello");
}

class homotopy.Homotopy(language)

Facade class for accessing homotopy tool.

add_lib_folder(path)

Add path to a snippet library

Parameters:path – Path

compile(snippet_text)

Compile a snippet.

Parameters:snippet_text – Snippet text Returns:Compiled snippet text

Console application

Homotopy can also be used as a console application. In fact, this is the intended way of using is via editor plugins.

C:\dev\homotopy>homotopy -h
usage: homotopy [-h] [-t N] [-c] [-p PATH] language snippet

Compile a snippet.

positional arguments:
  language              Language for the snippet to be compiled to
  snippet               A snippet to be compiled

optional arguments:
  -h, --help            show this help message and exit
  -t N, --tabsize N     Number of spaces in one tab. Tabs remain tabs if
                        absent
  -c, --cursor          Indicate cursor marker in compiled snippet
  -p PATH, --path PATH  Path to snippet library folders separated by ::