Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

segfault in [box] demo in sixel environments #2546

Closed
dankamongmen opened this issue Jan 10, 2022 · 14 comments
Closed

segfault in [box] demo in sixel environments #2546

dankamongmen opened this issue Jan 10, 2022 · 14 comments
Assignees
@dankamongmen
Labels
bitmaps bitmapped graphics (sixel, kitty, mmap) bug Something isn't working
Milestone
3.1.0

Comments

@dankamongmen
Copy link
Owner

Saw this in both xterm and contour, the same stack on both. 3.0.5-prerelease.
@dankamongmen dankamongmen added bug Something isn't working bitmaps bitmapped graphics (sixel, kitty, mmap) labels Jan 10, 2022
@dankamongmen dankamongmen added this to the 3.1.0 milestone Jan 10, 2022
@dankamongmen dankamongmen self-assigned this Jan 10, 2022
@dankamongmen
Copy link
Owner Author

i'm having trouble reproducing this...wish i'd recorded more information, ugh =[.

@dankamongmen
Copy link
Owner Author

alright, gonna give it an ASAN run and close this if i can't reproduce. IIRC the segfault was down in sixel stuff, so maybe this was transient during that work.

@dankamongmen dankamongmen added the nonfix closed without a successful fix (invalid, wontfix) label Jan 12, 2022
@dankamongmen
Copy link
Owner Author

alright, closing it up

@dankamongmen
Copy link
Owner Author

just seen in contour:

==2895332==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62000015c040 at pc 0x7ff8003f47be bp 0x7ffce39d91e0 sp 0x7f8
READ of size 4 at 0x62000015c040 thread T0
    #0 0x7ff8003f47bd in sprixel_invalidate (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0xfe7bd)
    #1 0x7ff8003d0095 in rasterize_core (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0xda095)
    #2 0x7ff8003d2dff in notcurses_rasterize_inner (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0xdcdff)
    #3 0x7ff8003d4f5e in raster_and_write (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0xdef5e)
    #4 0x7ff8003dcdae in ncpile_rasterize (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0xe6dae)
    #5 0x5652ff06c13b in demo_render (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x3613b)
    #6 0x5652ff06c46f in demo_nanosleep_abstime_ns (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x3646f)
    #7 0x5652ff06c6f9 in demo_nanosleep (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x366f9)
    #8 0x5652ff08760d in slideitslideit (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x5160d)
    #9 0x5652ff089e81 in trans_demo (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x53e81)
    #10 0x5652ff054205 in main (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x1e205)
    #11 0x7ff8001177ec in __libc_start_main ../csu/libc-start.c:332
    #12 0x5652ff055d69 in _start (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x1fd69)

0x62000015c040 is located 96 bytes to the right of 3936-byte region [0x62000015b080,0x62000015bfe0)
allocated by thread T0 here:
    #0 0x7ff800678987 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7ff800417b0b in ncvisual_render_pixels (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+0x121b0b)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/dank/src/dankamongmen/notcurses/build/libnotcurses-core.so.3+

@dankamongmen dankamongmen reopened this Jan 21, 2022
@dankamongmen dankamongmen removed the nonfix closed without a successful fix (invalid, wontfix) label Jan 21, 2022
@dankamongmen
Copy link
Owner Author

reproducible, huzzah. run ./notcurses-demo -p ../data/ -c -m3 in

notcurses 3.0.4 on Contour 0.3.0-unreleased-master-115721fc (Linux 5.16.1nlb)
30 rows (22px) 130 cols (11px) 660x1430 rgb+256 colors

@dankamongmen
Copy link
Owner Author 

    #0 0x7f56cc6c663d in sprixel_invalidate /home/dank/src/dankamongmen/notcurses/src/lib/sprite.c:109
    #1 0x7f56cc6a6c04 in rasterize_core /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1222
    #2 0x7f56cc6a94ef in notcurses_rasterize_inner /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1290
    #3 0x7f56cc6aad5e in raster_and_write /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1341
    #4 0x7f56cc6b0fc4 in notcurses_rasterize /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1381
    #5 0x7f56cc6b0fc4 in ncpile_rasterize /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1523
    #6 0x55d87aa21b3b in notcurses_render /home/dank/src/dankamongmen/notcurses/include/notcurses/notcurses.h:1093
    #7 0x55d87aa21b3b in demo_render /home/dank/src/dankamongmen/notcurses/src/demo/hud.c:640
    #8 0x55d87aa21e6f in demo_nanosleep_abstime_ns /home/dank/src/dankamongmen/notcurses/src/demo/hud.c:559
    #9 0x55d87aa220f9 in demo_nanosleep /home/dank/src/dankamongmen/notcurses/src/demo/hud.c:575
    #10 0x55d87aa3ccfd in slideitslideit /home/dank/src/dankamongmen/notcurses/src/demo/trans.c:93
    #11 0x55d87aa3f6d5 in slidepanel /home/dank/src/dankamongmen/notcurses/src/demo/trans.c:264
    #12 0x55d87aa3f6d5 in trans_demo /home/dank/src/dankamongmen/notcurses/src/demo/trans.c:316
    #13 0x55d87aa0a205 in ext_demos /home/dank/src/dankamongmen/notcurses/src/demo/demo.c:226
    #14 0x55d87aa0a205 in main /home/dank/src/dankamongmen/notcurses/src/demo/demo.c:583
    #15 0x7f56cc40b7ec in __libc_start_main ../csu/libc-start.c:332
    #16 0x55d87aa0bd69 in _start (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x1fd69)

@dankamongmen
Copy link
Owner Author 

    #0 0x7fea1a45563d in sprixel_invalidate /home/dank/src/dankamongmen/notcurses/src/lib/sprite.c:109
    #1 0x7fea1a435c04 in rasterize_core /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1222
    #2 0x7fea1a4384ef in notcurses_rasterize_inner /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1290
    #3 0x7fea1a439d5e in raster_and_write /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1341
    #4 0x7fea1a43ffc4 in notcurses_rasterize /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1381
    #5 0x7fea1a43ffc4 in ncpile_rasterize /home/dank/src/dankamongmen/notcurses/src/lib/render.c:1523
    #6 0x556c0bfa6b3b in notcurses_render /home/dank/src/dankamongmen/notcurses/include/notcurses/notcurses.h:1093
    #7 0x556c0bfa6b3b in demo_render /home/dank/src/dankamongmen/notcurses/src/demo/hud.c:640
    #8 0x556c0bf9664c in box_demo /home/dank/src/dankamongmen/notcurses/src/demo/box.c:272
    #9 0x556c0bf8f205 in ext_demos /home/dank/src/dankamongmen/notcurses/src/demo/demo.c:226
    #10 0x556c0bf8f205 in main /home/dank/src/dankamongmen/notcurses/src/demo/demo.c:583
    #11 0x7fea1a19a7ec in __libc_start_main ../csu/libc-start.c:332
    #12 0x556c0bf90d69 in _start (/home/dank/src/dankamongmen/notcurses/build/notcurses-demo+0x1fd69)

so this is indeed the same problem as what we saw in box. good! one fewer mystery in the world, soon enough.

@dankamongmen
Copy link
Owner Author

INVALIDATING AT 20/41 (3/9) TAM: 5
INVALIDATING AT 20/42 (3/10) TAM: 5
INVALIDATING AT 20/43 (3/11) TAM: 5
INVALIDATING AT 20/44 (3/12) TAM: 5
INVALIDATING AT 19/44 (4/11) TAM: 5
INVALIDATING AT 19/45 (4/12) TAM: 6
INVALIDATING AT 19/46 (4/13) TAM: 5
INVALIDATING AT 19/48 (4/12) TAM: 6
INVALIDATING AT 19/49 (4/13) TAM: 5
INVALIDATING AT 20/37 (5/4) TAM: 6
INVALIDATING AT 20/44 (5/11) TAM: 5

almost certainly hitting an invalid cell in sprixel_invalidate().

@dankamongmen
Copy link
Owner Author

INVALIDATING AT 20/45 (5/12) is where we actually blow up

@dankamongmen
Copy link
Owner Author

that would be [5 * dimx + 12] into the TAM

ncplane_resize_internal:840:6x12 @ 30/30 → 6/12 @ 30/30 (want 6x12@0/0)

so dimx is 12 cells at 11 pixels each, 132 pixels == dimx.

5 * 132 == 660 + 12 == 672

6 cells tall at 22px each == 132 pixels tall. oh hey, it's a square. anyway that's 17424 auxvecs for the total image, and 242 auxvecs per cell.

hrmmm nothing leaping out yet...

@dankamongmen
Copy link
Owner Author

oh i'm stupid as hell, yeah 5/12 is no good on a 6/12 image, doofus, did you forget how to count from 0

@dankamongmen
Copy link
Owner Author

oh shit i bet it's dem margins!

@dankamongmen
Copy link
Owner Author

ok, yep, it doesn't prove anything, but running without -m3 does not hit this problem. tally-ho!

@dankamongmen
Copy link
Owner Author

yep, i think we've got a good fix! huzzah!

dankamongmen added a commit that referenced this issue Jan 21, 2022
@dankamongmen
[rasterize_core] pass margin-adjusted location to sprixel_invalidate #…
a6fc8f0 
…2546
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dankamongmen dankamongmen

Labels
bitmaps bitmapped graphics (sixel, kitty, mmap) bug Something isn't working
Projects
None yet
Milestone
3.1.0
Development

No branches or pull requests
1 participant
@dankamongmen