Monday, May 20, 2013

Powerful insert mode completion configuration for Vim paired with AutoComplPop

Combine results of multiple sources for completion into one menu that pops up automatically. To keep the noise low you can add to an open Popup Menu at different leading word lengths and you will be able to complete matches with the same key (default: Tab) in forward and backward menus (e.g.: <C-P>).

The most interesting part is the localcomplete#localMatches function which allows you to scan an area around the current line (up to the whole buffer) and configure how to sort matches. My favorite sort order starts at the current line and alternates matches from above and below. You can combine local completion with omni completions like RopeOmni for Python.

Local, Rope, and all-buffers completion combined

Now a little disclaimer. This is partially hacking and you might have to wrestle with some quirks. Vim's Popup Menu and completion functions aren't easily accessible. However, I mostly got what I wanted and it is actually a considerable improvement to my workflow. Give it a try.  :D


Generally the scheme in all this is that there are never items preselected in the menu. That is also the most useful way to compare omni completion methods. In addition to what follows, you can still configure completion methods and compare results through the explicit triggers <C-X><C-U> and <C-X><C-O>.
autocmd FileType python setlocal omnifunc=RopeOmni
autocmd FileType python setlocal completefunc=jedi#complete

Beware of the value longest in the completeopt Vim setting. I don't use it, but I've noticed that it conflicts with the expected behavior.

If you don't know much about insert mode completion, reading *ins-completion* in the Vim docs will help you understand some of the terms (and restrictions of the implementation) that follow.

AutoComplPop is used to trigger the Popup Menu. I did not look much at YouCompleteMe. ACP looked like it was what I needed. In particular, you can pretty much configure exactly what will happen for each individual file type. Nevertheless, you will probably be able to use the completion functions with YCM.


The first thing you need is a patched AutoComplPop. Please read the commit messages on top of the fork-point to see what changed. I also updated the documentation in the fork.

The most notable change is that you can configure the keys used to browse the menu. Furthermore the colors and mappings are different for reverse (e.g. <C-P>) and forward menus. This gives you a clue where to look for the most likely completion, and you can always use the same key for completion. It is possible to turn off the mapping switch, and you can configure the same color for both directions.

When you use the switch (the default) you need to include a menu adaption into insert mode completion mappings. I tried to avoid it but when done inside ACP, the previous color was always flickering through.

Here some examples. The number in the call is the direction. One is forward, and two is backward.
" Filename completion
inoremap <C-F> <C-R>=acp#pum_color_and_map_adaptions(1)<CR><C-X><C-F><C-P>

" Spelling corrections
inoremap <C-S> <C-R>=acp#pum_color_and_map_adaptions(1)<CR><C-X><C-S><C-P>

" Omni completion
inoremap <C-O> <C-R>=acp#pum_color_and_map_adaptions(1)<CR><C-X><C-O><C-P>

" User defined completion
inoremap <C-U> <C-R>=acp#pum_color_and_map_adaptions(1)<CR><C-X><C-U><C-P>

" Switch to keyword completion backwards
inoremap <C-L>  <C-R>=acp#pum_color_and_map_adaptions(2)
            \ <CR><C-R>=pumvisible()
            \ ? "\<lt>C-E>\<lt>C-P>\<lt>C-N>"
            \ : "\<lt>C-P>"<CR>

" Close the pum with <Return>.  Otherwise, it would restore the old value after
" manual completion.
inoremap <Return> <C-R>=pumvisible()
            \ ? "\<lt>C-X>\<lt>Return>"
            \ : "\<lt>Return>"<CR>

" Allow to exit the pum with Up/Down
inoremap <Up>  <C-R>=pumvisible() ? "\<lt>C-X>\<lt>Up>" : "\<lt>Up>"<CR>
inoremap <Down>  <C-R>=pumvisible() ? "\<lt>C-X>\<lt>Down>" : "\<lt>Down>"<CR>

A few hints for the ACP configuration.  See the ACP doc for the actual configuration help.
" Remove dictionary lookup from the Vim keyword completion.  It did always
" complete the first match for me.  If you edit files with tags you might
" want to add those.
let g:acp_completeOption = '.,w,b'

" How keyword completion is triggered.  Usually you want variables before
" the current line.  ... Unless you write a file bottom up, that is.
let g:acp_behaviorKeywordCommand = "\<C-P>"

" This adds the local-completion function before all other completions
" (snipmate, keyword, file) in the g:acp_behavior records set up by ACP.
let g:acp_behaviorUserDefinedFunction = 'localcomplete#localMatches'
let g:acp_behaviorUserDefinedMeets = 'acp#meetsForKeyword'

And templates for some new options. All given values are the defaults
" Highlights to use for the pop up menu.
let g:acp_colorForward = 'AutoComplPopColorDefaultForward'
let g:acp_colorReverse = 'AutoComplPopColorDefaultReverse'

" Specify the keys used to select entries.  These create insert mode
" mappings like you know them from the Vim documentation.
let g:acp_nextItemMapping = ['<TAB>', '\<lt>TAB>']
let g:acp_previousItemMapping = ['<S-TAB>', '\<lt>S-TAB>']

" Turn the reverse mapping switch on/off:
let g:acp_reverseMappingInReverseMenu = 1

" Don't preselect the first item.  Use the same key to select and to browse
let g:acp_autoselectFirstCompletion = 0


The localcomplete plugin tries to replicate some built-in Vim completions so that they can be combined and configured.

Of interest are essentially the following two files:


Here you find three completion functions:

This is the function mentioned above. It searches through the configured area of the current buffer only. All of the configuration in the first section of the source file applies to it. Please read through that section. There is no documentation file.

This function searches through all buffers. It respects the case and keyword-char configuration. It's just an unsophisticated replacement for the functionality provided by Vim.

Search the dictionary configured in Vim's 'dictionary' setting for matches. All matches are done case-sensitively, because otherwise all the names in the dictionary would come first. It is a very simple function. The dictionary has to be utf-8 encoded.

All three functions can have individual minimum leading word lengths configured after which they start to produce results. This can be used to keep the number of matches in the menu reasonable. For example, you might want to add local-matches after two characters, but not the masses of dictionary matches at that point. The variables for that are described in the second section of this file.


This is a pretty rough and hardcoded module for demonstration purposes.

It contains a function to combine multiple completion functions into one. ACP can be configured to try several completion functions in a row until one generates results, but it is much more useful to have multiple results combined into one menu. Here is a client function that calls it with the three completion functions from above.
function combinerEXP#completeCombinerTextish(findstart, keyword_base)
    let l:all_completers = [
                \ 'localcomplete#localMatches',
                \ 'localcomplete#allBufferMatches',
                \ 'localcomplete#dictMatches',
                \ ]
    return combinerEXP#completeCombinerABSTRACT(
                \ a:findstart,
                \ a:keyword_base,
                \ l:all_completers
                \ 0)
The zero is an index into l:all_completers to select the function used in the findstart mode. Because of the way completion functions interact with Vim, there can be only one findstart column that is shared by all combined completion functions.

There is a Python combiner that addresses requirements when combining other completion functions with RopeOmni from python-mode. If you are interested in rope completions, you should take a closer look at this file. Here is an example combiner:
function combinerEXP#completeCombinerPython(findstart, keyword_base)
    let l:before_rope = []
    let l:after_rope = [
                \ 'localcomplete#localMatches',
                \ 'localcomplete#allBufferMatches',
                \ ]
    return combinerEXP#ropeCombiner(
                \ a:findstart,
                \ a:keyword_base,
                \ l:before_rope,
                \ l:after_rope,
                \ 0)
It makes sense to prepend local-completion with a very slim search area. The all-buffers completion searches the current buffer first, and so you can fall back to that completion for all matches in the current buffer. More on that below.

Configuring Content

Alright. Now we can connect those two plugins and get a lot of opportunities for configuration. You can specify the completion for each &filetype individually by filling in the g:acp_behavior variable.

For each filetype you can configure multiple entries, that are tried in turn until one generates matches. The following example creates the entry for all files that don't have a filetype specific entry (e.g. an 'xml' key instead of the '*'). This one tries the text combiner first, and falls back to keyword completion in reverse mode.
if ! exists("g:acp_behavior")
      let g:acp_behavior = {
        \ '*' : [
        \     {
        \       'completefunc': 'combinerEXP#completeCombinerTextish',
        \       'command': "\<C-X>\<C-U>",
        \       'meets': 'acp#meetsForKeyword',
        \       'repeat': 1,
        \     },
        \     {
        \       'command': "\<C-P>",
        \       'meets': 'acp#meetsForKeyword',
        \       'repeat': 0,
        \     },
        \ ],

Here are the two completions side by side.

Localcomplete and standard keyword completion side-by-side (the sorting is different)

Now let's configure at what point which entries appear.

It is important to note that ACP has a minimum leading word requirement, as well as the completion functions from localcomplete itself. ACP won't open the menu unless the minimum length is reached. These values are used in the corresponding 'meets' functions. You can see one of those above (acp#meetsForKeyword). You can write your own meets functions. They are called with the line content before the cursor position and are supposed to return non-zero if the requirements for opening the Popup Menu are met.
" count of chars required to start keyword completion
let g:acp_behaviorKeywordLength = 2

If you have multiple completion functions from the localcomplete plugin combined into one, you can configure individual minimal lengths of the word before the cursor. There is a catch, however. The Popup menu has to be updated to display the new content. You can do that after every typed character, but you'll soon notice some lag when combined with expensive omni completions. Therefore it is possible to configure particular word-lengths at which the menu should be updated. You can simply use the minimum length variables for that:
let g:localcomplete#LocalMinPrefixLength = 1
let g:localcomplete#AllBuffersMinPrefixLength = 1
let g:localcomplete#DictMinPrefixLength = 5

let g:acp_refeed_checkpoints = [
            \ g:localcomplete#LocalMinPrefixLength,
            \ g:localcomplete#AllBuffersMinPrefixLength,
            \ g:localcomplete#DictMinPrefixLength]

" Beware. Probably expensive (flickering)
"let g:acp_refeed_after_every_char = 0
In the same way you can transmit the additional keyword configuration to ACP:
let g:localcomplete#AdditionalKeywordChars = '-'
let g:acp_keyword_chars_for_checkpoint =
            \ g:localcomplete#AdditionalKeywordChars

Python Configuration

Here is part of the Python configuration I am playing with right now. A very slim area is searched before RopeOmni has its turn. I think it has a better chance of putting local variables in the right order.
let b:LocalCompleteLinesAboveToSearchCount = 15
let b:LocalCompleteLinesBelowToSearchCount = 10
The combiner for it looks like this:
function! g:localFirstPythonCombiner(findstart, keyword_base)
    let l:before_rope = [
                \ 'localcomplete#localMatches',
                \ ]
    let l:after_rope = [
                \ 'localcomplete#allBufferMatches',
                \ ]
    return combinerEXP#ropeCombiner(
                \ a:findstart,
                \ a:keyword_base,
                \ l:before_rope,
                \ l:after_rope,
                \ 0)
We need to tell ACP when to add entries to the menu. This is really lagging when done after every typed character.
" Minimum leading word lengths
let b:LocalCompleteLocalMinPrefixLength = 3
let b:LocalCompleteAllBuffersMinPrefixLength = 5

" Restart omni completion after these word lengths.
let b:acp_refeed_checkpoints = [
            \ b:LocalCompleteLocalMinPrefixLength,
            \ b:LocalCompleteAllBuffersMinPrefixLength,
            \ ]

" Preemptively override global values
let b:acp_refeed_after_every_char = 0
let b:LocalCompleteAdditionalKeywordChars = ''
let b:acp_keyword_chars_for_checkpoint =
            \ b:LocalCompleteAdditionalKeywordChars
In the RopeOmni version that I used is a bug where keywords override object attributes. So, for example, when you add local-matches after two characters, and an object attribute starts with either of 'is in as', all initially good rope results are removed from the menu. Therefore I would try to avoid using only small checkpoints without a bigger one.

You can add all this in a ftplugin/python.vim file so that it is nicely grouped. Here is the ACP configuration for Python that uses the combiner from above:
let g:acp_behavior['python'] = [
        \     {
        \       'command': "\<C-X>\<C-U>",
        \       'completefunc': 'g:localFirstPythonCombiner',
        \       'meets': 'acp#meetsForPythonOmni',
        \       'repeat': 1,
        \     },
The repeat dictionary key is important for omni-completions that can give you info about, for example, function parameters after typing an opening parenthesis. It tries to restart the completion menu when it closes. If that doesn't work it's probably the fault of the meets function. Take a look at the ones defined in ACP if you want to write your own.

The startup word length for this particular meets function can be set with the following variable. I've added dots and opening parentheses to that function so that a setting of 1 will give you maximal info about objects while not always bothering you.
let g:acp_behaviorPythonOmniLength = 1

Display function parameters and default arguments after an opening parenthesis

RopeOmni needs a syntactically correct file. By default you see that the cursor becomes red when RopeOmni errors are caught in ropeCombiner. Anyway, the other completions can jump in. And with the error silencing done in the combiner it is actually a quiet good and fun completion.

Prefer surrounding lines over Rope's object attributes


Snipmate has the Tab key hardcoded, but you can easily find and change it in after/plugin/snipMate.vim.

This is the standard entry that gets normally added if you don't override entries in g:acp_behavior:
{ 'meets': 'acp#meetsForSnipmate',
\ 'completefunc': 'acp#completeSnipmate',
\ 'onPopupClose': 'acp#onPopupCloseSnipmate',
\ 'repeat': 0, 
\ 'command': "\<C-X>\<C-U>"},
I would recommend to put it first and I think it works best when the length limit is lower than the others:
let g:acp_behaviorSnipmateLength = 1
Only triggers that start with an uppercase character end up in the Popup Menu and you have to explicitly select the completion for it to expand (default: <C-Y>)


Now the sad part. ACP is a strange beast. There is no real access to Vim's PopUp Menu and this is all done by streaming a bunch of key presses to Vim. I don't understand everything that is happening sometimes, but here a few tips:

For the localcomplete functions, always set repeat to 1 (or let it unspecified). If the updates at checkpoints don't work, this might be the cause.  However, the completion is quiet persistent in that case.  If you don't like that, try to increase or decrease the checkpoint by one.
Another alternative in this case is to create mappings that turn the auto-completion on/off:
map <S-F5> :AcpDisable<CR>
map <F5>   :AcpEnable<CR>

If you want to trigger built-in commands, I would recommend to set repeat to 0 explicitly. Otherwise, if you use checkpoints, the completion is undone to the point where the PopUp Menu was opened.

ACP prepares individual records for the following filetypes: ruby, python, perl, xml, html, xhtml, css, javascript, coffee, ls.
This can be a bit surprising if you edit any of such files infrequently. The most failsafe way to fall back to the default behavior is probably to add an autocommand like this:
autocmd FileType ruby call remove(g:acp_behavior, "ruby")

I hope I didn't break too much for you, but I am done experimenting with it. It would be awesome if there could be a more sophisticated access to Vim's PopUp Menu. That would make things a lot easier, I guess.

But don't let all this discourage you. I am actually quiet happy with it. In particular in Python.

Give <C-P> some relief

Happy playing around with the word lengths. There must be some optimal adjustment.

(The snippets from the source code are under LGPLv3)