Skip to content

Add RSS memory status collecting routine to get RSS per tests and workers #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

avtikhon
Copy link
Contributor

@avtikhon avtikhon commented Apr 13, 2021

Patch set consists of 2 commits:

  1. Add RSS memory status collecting routine

    Added RSS memory status collecting routine which parses file:

    /sys/fs/cgroup/memory/memory.stat

    for RSS value and run this routine after each sent test lua command in
    listeners. All results it collects in main tests worker log file:

    var/log/<worker name>.log

    For this routine use added test-run new option:

    --collect-statistics

    which should be used to use this routine.

    Closes resources: flaky issues on lack of memory tarantool-qa#98

  2. Enable logging TAP tests on each testing condition

    Found that TAP test do not print any information on tested conditions
    in tests worker log file. It can be used to check test stages for
    data written into this log file.

@avtikhon avtikhon self-assigned this Apr 13, 2021
@avtikhon avtikhon added the teamQ label Apr 13, 2021
@avtikhon avtikhon force-pushed the avtikhon/gh-qa-98-rss-collect branch 3 times, most recently from 6525c15 to 452c434 Compare April 13, 2021 05:03
@avtikhon avtikhon requested a review from Totktonada April 13, 2021 11:07
@kyukhin kyukhin linked an issue Apr 16, 2021 that may be closed by this pull request
@avtikhon avtikhon force-pushed the avtikhon/gh-qa-98-rss-collect branch from d03d3fd to 17d2b6b Compare April 27, 2021 07:42
@avtikhon avtikhon added the do not merge Not ready to be merged label Apr 27, 2021
@avtikhon avtikhon force-pushed the avtikhon/gh-qa-98-rss-collect branch from 17d2b6b to dd030dd Compare April 29, 2021 13:16
@avtikhon avtikhon requested a review from kyukhin April 30, 2021 11:34
Added RSS memory status collecting routine which parses file:

  /sys/fs/cgroup/memory/memory.stat

for RSS value and run this routine after each sent test lua command in
listeners. All results it collects in main tests worker log file:

  var/log/<worker name>.log

For this routine use added test-run new option:

  --collect-statistics

which should be used to use this routine.

Closes tarantool/tarantool-qa#98
Found that TAP test do not print any information on tested conditions
in tests worker log file. It can be used to check test stages for
data written into this log file.
@avtikhon avtikhon force-pushed the avtikhon/gh-qa-98-rss-collect branch from dd030dd to d9d7124 Compare April 30, 2021 16:21
@avtikhon avtikhon added do not merge Not ready to be merged and removed do not merge Not ready to be merged labels Apr 30, 2021
@Totktonada
Copy link
Member

I clarified the task: I want a built-in tool that will spot tests that use large amount of memory and will be enabled always: locally and in CI. So it should not be under an option and should be cheap enough (not after each command).

Sure, we may want a tool that will allow in-depth analysis, but it is not my primary wish. It is also possible that we'll get enough information in test-run logs after #298: we'll see commands and samples on the same timeline.

@kyukhin kyukhin closed this May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge Not ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

resources: flaky issues on lack of memory Add memory usage profiling for running tests
3 participants