Much more powerful gq replacement function #29

JelteF · 2014-07-08T20:22:14Z

This fixes #27 and contains the commit from #28 as well.
It contains quite a lot of edge cases, like when there is no splitting possible and when trying to reformat docstrings.
I'm not sure how to write tests for this because it would have to test for something different than the other tests.

hynek · 2014-07-27T08:28:36Z

Playing with it, it does this:

def f():
    f = ("safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj ")

to

def f():
    f =
    ("safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj ")

Trying to format

def f():
   """
   safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj 
   """

in its entirety leads to

def f():
   """ safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj """

Could you explicitly spell out what this is supposed to achieve again? :)

Thanks again!

blueyed · 2015-05-06T16:14:02Z

@JelteF
Any update on this?

blueyed · 2015-05-06T16:22:08Z

indent/python.vim

+        return 1
+    endif
+
+    if len(getline(a:lnum)) < 80 && l:count == 1 " No need for gq


That should probably use &textwidth + 1 instead of 80?

blueyed · 2015-05-06T16:26:08Z

I'm not sure how to write tests for this because it would have to test for something different than the other tests.

What do you mean?
There are tests for gq already, which could be used as a base.

JelteF · 2015-05-08T11:24:28Z

I forgot about this a bit. More up to date code, that fixes some bugs, is over at https://github.com/pangloss/vim-javascript
I currently don't have a lot of time to do a lot of open source stuff. If you want you could fork from my repo and apply the changes you mentioned plus the fixes done over at the javascript plugin.
Some links to specific commits/issues, that could be useful:
pangloss/vim-javascript#229
pangloss/vim-javascript@18106a3

blueyed · 2016-09-03T01:28:41Z

@JelteF
Any update?

…y through

JelteF · 2016-09-06T21:34:05Z

I just updated the code based on the version that was available in vim-javascript until recently.
To be clear of the point of this branch is that gq does not break on spaces inside single line strings.
The following snippet is what currently works and what doesn't:

# currently working

(222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo')
a = list('ueauea euaeouaeo uea', 'ueaeu eau oeuao ', 'ueaue eauoeu ', 'ueau uaeu aeuaeoua')
# ueau  euaeou  euaeo ua eou aeou eao ueo uaeo u eo u eaoueo ua eou aeo u eaou aeo ue a uoe uaoe u aeou

def a():
    return ('safdj asldfj  öalsdfj ' 'sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj ' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

# currently not working
def b():
    # A check should be added to see if the split position is between brackets
    # otherwise it should cancel
    return ('safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj q')

# Here no split position is available, for some reason the cursor moves to the first column after pressing gqq instead of doing nothing.
('u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou euauea ouaoe aaaaaaaaa aaayxza  uuu uubc')

def c():
    # docstrings should be done with normal gq like comments

    """ safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj """

I'm not very experienced with vimscript, so if you know of a simple way or a pointer to fix one of the three bugs below please let me know.

blueyed · 2016-09-06T22:11:50Z

indent/python.vim

+endfunction
+
+function! GetPythonPEPFormat(lnum, count)
+  let l:tw = &textwidth ? &textwidth : 79


Should be 78 by default IIRC.

The maximum line length according to PEP8 is 79. https://www.python.org/dev/peps/pep-0008/#maximum-line-length

blueyed · 2016-09-06T22:18:04Z

It might be worth adding some tests for this.. :)

For the bugs, I recommend calling it manually using :debug call GetPythonPEPFormat(…) and then step through what it is doing (use s, n, echo, etc).

JelteF · 2016-09-06T22:37:33Z

Tests are definitely needed. I'll add them later when I have some time again. I'll try to fix the bugs then as well.

JelteF · 2016-09-11T19:15:30Z

I fixed the issues you had with the pull request. Creating tests is not as simple as I would have thought, because mine does not require checking the current indent, but instead the place where the indent took place.

JelteF · 2016-09-11T20:30:33Z

All bugs that I found myself are fixed now:

These are the tests I'm currently manually doing and all of them are sane.

(222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo')
[222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo']
{222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo'}
a = list('ueauea euaeouaeo uea', 'ueaeu eau oeuao ', 'ueaue eauoeu ', 'ueau uaeu aeuaeoua')
# ueau  euaeou  euaeo ua eou aeou eao ueo uaeo u eo u eaoueo ua eou aeo u eaou aeo ue a uoe uaoe u aeou

def a():
    return ('safdj asldfj  öalsdfj ' 'sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj ' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

def b():
    a = ['safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj q']
    b = {'safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj q'}
    return ('safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj q')

# Here no split position is available
('u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou euauea ouaoe aaaaaaaaa aaayxza  uuu uubc')

def c():
    # docstrings should be done normally like regular strings

    """
    safdj asldfj  öalsdfj sdölj aölfj asdöflj asdöflj asölfj asdölfj asdölfj  dsöalfj
    """

JelteF · 2016-09-20T21:01:01Z

I just updated the pull request again with the new feature that allows breaking in strings to return valid python. I did break some of my tests again though and added some new ones:

(222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo')
[222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo']
{222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo'}
a = list('ueauea euaeouaeo uea', 'ueaeu eau oeuao ', 'ueaue eauoeu ', 'ueau uaeu aeuaeoua')
# ueau  euaeou  euaeo ua eou aeou eao ueo uaeo u eo u eaoueo ua eou aeo u eaou aeo ue a uoe uaoe u aeou

def a():
    return ('safdj asldfj  oalsdfj ' 'sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj ' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

def a():
    return ('safdj asldfj  oalsdfj asdoflj asolfj asdolfj asdolfj  dsoalfj ' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

def b():
    a = ['safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q']
    b = {"safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q"}
    # This one is broken (infinite loop)
    c = {"uuaeuaoeuaeouaeouaeosafdjasldfjoalsdfjsdoljaolfjasdofljasdofljasolfjasdolfjasdolfjdsoalfjq"}

    return ('safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q')

('u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e aueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc')
("u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc")

def c():
    # docstrings should be done normally like regular strings

    """
    safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj dsoalfj
    """

# These are broken (test for multiline is broken)
def b():
    a = ['safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q']
    b = {"safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q"}
    # This one is broken (infinite loop)
    c = {"uuaeuaoeuaeouaeouaeosafdjasldfjoalsdfjsdoljaolfjasdofljasdofljasolfjasdolfjasdolfjdsoalfjq"}

    return ('safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q')

JelteF · 2016-09-21T22:37:43Z

I made gq much more powerful with the latest changes. It now not only always results in valid python code, but it makes it look quite good automatically as well. Please just try the test file below with the version of the library in this pull request to see what I mean.

There is now only one bug left that I know of (and it is not that big of a problem). One of the bugs was fixed by changing python-syntax, so this updated version is needed for everything to work: hdima/python-syntax#55

The current testing method in this repository is hard to change for the tests that would be needed for this feature. It would probably be better to have an input file and do gqq on certain lines and make sure the output file is what is expected after these operations. Here is the updated file I use for manual testing at the moment, with the only bug that's left at the end (it contains a lot of edgecases, but some are also just double):

(222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo')
[222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo']
{222222222222222222222222, 'aaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa aaayxza  uuu uubce uae aeuoe eaueo'}
a = list('ueauea euaeouaeo uea', 'ueaeu eau oeuao ', 'ueaue eauoeu ', 'ueau uaeu aeuaeoua')
# ueau  euaeou  euaeo ua eou aeou eao ueo uaeo u eo u eaoueo ua eou aeo u eaou aeo ue a uoe uaoe u aeou

def a():
    return ('safdj asldfj  oalsdfj ' 'sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj ' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

def a():
    return ('safdjaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaauaeuaeouaeouaeou' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')

def b():
    a = ['safdj asldfj  oalsdfj sdolj aolfj sdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q']
    b = {"safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q"}

    return ('safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q')

('u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e aueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc')
("u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc")

def c():
    # docstrings should be done normally

    """
    safdj iü asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj dsoalfj
    """


def b():
    a = ['safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q', 'ueaueaou']
    b = {"safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q"}
    return ('safdj asldfj  oalsdfj sdolj aolfj asdoflj asdoflj asolfj asdolfj asdolfj  dsoalfj q')

ceeuaeouaeu = {"uuaeuaoeuaeouaeouaeosafdjasldfjoalsdfjsdoljaolfjasdofljasdofljasolfjasdolfjasdolfjdsoalfjq", 'ueaeuoaeou eaueouaeouaeou'}
abc = "ueauoe eauoeau eaueoua eouaeo uaoeuaeou euaeo uaoe uaoeua euaeou eou aoeu aoeu aoeu aeoua oeu aoeu aoeuaeouaeou aeouaoeuaoeua"

def a():
    return ('safdjaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaauaeuaeouaeouaeou' 'euaeouaeuaeoua eau eou ae uae ouaeou aeou aeou ' 'ueaue eaueuaeuaeoueo eaueoua euaeouaeouao ')


(r'u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e aueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc')
(r"u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc")
(b'u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e aueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc')
(b"u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa uea e eaueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc")

('üüüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü ueauaeouaeouue üüüüüüüüüuuuuu')
('üüüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü')
('uuuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu')
('uuuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuuuuuuuu uuuuuuuuu uuuuuuuuu uuuuuuuuu')
('üüüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü uüüüüüüüüüüüüüü üüüüüüüüü üüüüüüüüü üüüüüüüüü')

# BUG: eats spaces directly after the breakpoint
("u eua eua uae oueao ueo ueoa ueo uaeo aaaaaaaaaaaaaaaaaaaaaaaaa                               uea e eaueou uauea ouaoe aaaaaaaaa aaayxza  uuu uubc")

blueyed · 2016-09-21T23:06:25Z

This looks awesome!

What do you think about using https://github.com/junegunn/vader.vim for new tests?
I use it in several projects already, and could help with setting it up.
We can keep the existing tests as they are, but I also found it always hard to work with them.

blueyed · 2016-09-21T23:15:35Z

There is now only one bug left that I know of (and it is not that big of a problem). One of the bugs was fixed by changing python-syntax, so this updated version is needed for everything to work: hdima/python-syntax#55

How does it behave with the syntax file from Vim?
How are they different? https://github.com/vim/vim/blob/master/runtime/syntax/python.vim

blueyed · 2016-09-21T23:17:59Z

If it's about docstring strings, those are in the official syntax file: https://github.com/vim/vim/blob/master/runtime/syntax/python.vim#L112-L124 (IIRC the patch comes from or at least was driven by me.. ;))

JelteF · 2016-09-22T00:00:28Z

The vader testing seems much better suitable indeed. If you could set it up
that would be great.

As for the syntax file, I needed to be able to differentiate between
regular strings and multiline strings by using synattr, since different gq
behaviour is needed. Looking at the regular python syntax file it should be
easy to modify the code to work with that as well if you can request the
name of the matchgroups instead of the name with synIDattr.

JelteF · 2016-09-22T06:29:49Z

I just checked, synIDattr can't return matchgroups, only name and a lot of styling stuff.
So, the official syntax would also have to be changed to include multiline in the name.

blueyed · 2016-10-06T22:42:49Z

The vader testing seems much better suitable indeed. If you could set it up that would be great.

Will come back to this eventually, but I am lacking the time and energy for this currently.

pythonTripleQuotes is available on the quotes itself, using :echo reverse(map(synstack(1, 1), 'synIDattr(v:val,"name")')) on a file starting with """.
(It's not available in the string however)
You could therefore check if it's a string in general and then use searchpairpos('"""', '', '"""', 'b') to find it for the check - although that does not guarantee it to be the same string then - but maybe some of the existing methods can help with that.

blueyed · 2018-07-26T09:35:58Z

Please rebase this on master, which should give us a coverage report.

JelteF · 2018-07-26T10:52:51Z

I'm going to close this. I don't use it myself anymore, I've been using black for a while now to sortof get the same result.

JelteF mentioned this pull request Jul 9, 2014

Fix gq to not split strings halfway through pangloss/vim-javascript#201

Merged

blueyed added the enhancement label May 6, 2015

blueyed reviewed May 6, 2015
View reviewed changes

JelteF force-pushed the gq-fix branch from e09f1d9 to 77178ce Compare September 6, 2016 17:34

Add a start of a gq implementation that does not split strings halfwa…

bb87bed

…y through

JelteF force-pushed the gq-fix branch from 77178ce to bb87bed Compare September 6, 2016 20:41

blueyed reviewed Sep 6, 2016
View reviewed changes

JelteF added 3 commits September 11, 2016 20:29

Remove extra cursor call

630fdc7

Fallback to normal gq when no better solution is found

a9ff2d9

Fix trailing whitespace break

0f0eeb8

JelteF force-pushed the gq-fix branch from 5207752 to a9ff2d9 Compare September 11, 2016 18:39

Add normal mode to feedkeys

ed8ce61

JelteF added 2 commits September 11, 2016 22:24

Remove echom calls

1259823

Add check to see if space is inside brackets and add a "\" accordingly

ea4f96d

JelteF added 3 commits September 20, 2016 22:16

Add support for breaking inside strings

7b21d10

Use custom gq more often

0d22c60

Fix breaking in string for double quote strings

c8de03f

JelteF mentioned this pull request Sep 21, 2016

Add multiline to syntax info hdima/python-syntax#55

Open

JelteF added 8 commits September 21, 2016 22:12

Fix multiline string detection by using syntax plugin

d4b530f

Fix some annoying bugs

4f6697d

Fix quote choosing for special strings (r/u/b)

8801fc9

Fix string exclusion to include byte strings

d77a6cc

Add a bracket intsead of backslash + newline when it can help

99d3c50

Try adding newline after bracket instead of using backslash

85d31e2

Fix unicode support partially

146ba03

Fix unicode related bugs by using virtcol

58c5347

JelteF force-pushed the gq-fix branch from a0e68d0 to 58c5347 Compare September 21, 2016 22:25

JelteF changed the title ~~Fix for using gq in a string with spaces~~ Much more powerful gq replacement function Sep 21, 2016

JelteF added 3 commits September 22, 2016 09:44

Add support for strings with strformat and doctests

5391f8d

Use exclamation mark functions

05464ad

Remove gq<movement> code, it works fine without it

21ea58a

Fix bug after multiline removal

7445a12

JelteF closed this Jul 26, 2018

Much more powerful gq replacement function #29

Much more powerful gq replacement function #29

Uh oh!

Conversation

JelteF commented Jul 8, 2014

Uh oh!

hynek commented Jul 27, 2014

Uh oh!

blueyed commented May 6, 2015

Uh oh!

blueyed May 6, 2015

Choose a reason for hiding this comment

Uh oh!

blueyed commented May 6, 2015

Uh oh!

JelteF commented May 8, 2015

Uh oh!

blueyed commented Sep 3, 2016

Uh oh!

JelteF commented Sep 6, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueyed Sep 6, 2016

Choose a reason for hiding this comment

Uh oh!

JelteF Sep 6, 2016

Choose a reason for hiding this comment

Uh oh!

blueyed commented Sep 6, 2016

Uh oh!

JelteF commented Sep 6, 2016

Uh oh!

JelteF commented Sep 11, 2016

Uh oh!

JelteF commented Sep 11, 2016

Uh oh!

JelteF commented Sep 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelteF commented Sep 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blueyed commented Sep 21, 2016

Uh oh!

blueyed commented Sep 21, 2016

Uh oh!

blueyed commented Sep 21, 2016

Uh oh!

JelteF commented Sep 22, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JelteF commented Sep 22, 2016

Uh oh!

blueyed commented Oct 6, 2016

Uh oh!

blueyed commented Jul 26, 2018

Uh oh!

JelteF commented Jul 26, 2018

Uh oh!

Uh oh!

JelteF commented Sep 6, 2016 •

edited

Loading

JelteF commented Sep 20, 2016 •

edited

Loading

JelteF commented Sep 21, 2016 •

edited

Loading

JelteF commented Sep 22, 2016 •

edited

Loading