Add overhead count in heap stats #7917

deepikabhavnani · 2018-08-28T19:02:28Z

Description

Heap statistics are used for analyzing heap stats, but it doesn't tell anything about real heap usage or malloc overheads. Adding overhead_size element will help users to get the real heap usage.

Pull request type

[X] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Breaking change

deepikabhavnani · 2018-08-30T15:47:05Z

Resolves: #7343

deepikabhavnani · 2018-08-31T15:28:25Z

~~Needs work: Remove the overhead from mbed-os, we can use the allocator information to get the actual heap sizes.~~

deepikabhavnani · 2018-09-05T20:59:39Z

Updated overhead_size element to include the stats and allocator overhead.

deepikabhavnani · 2018-09-12T22:03:30Z

@c1728p9 @SeppoTakalo - Please review

SeppoTakalo · 2018-09-13T19:12:45Z

platform/mbed_alloc_wrappers.cpp

+
+typedef struct  {
+    size_t size;
+}mbed_heap_overhead_t;


Does this apply to all allocators or just one particular library? dlmalloc based?

Does not apply to all allocators. But tested with dlmalloc and all three tool chains

Which from Mbed OS point of view is "all"

Thanks..

cmonr · 2018-09-29T02:42:47Z

/morph build

mbed-ci · 2018-09-29T03:56:39Z

Build : SUCCESS

Build number : 3190
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7917/

Triggering tests

/morph test
/morph export-build
/morph mbed2-build

mbed-ci · 2018-09-29T06:09:15Z

Test : SUCCESS

Build number : 2992
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7917/2992

mbed-ci · 2018-09-29T06:15:35Z

Exporter Build : SUCCESS

Build number : 2784
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7917/

0xc0170 · 2018-10-01T15:26:41Z

We need to restart the tests (reports success but there is a hard fault in one of the tests), will restart

theotherjimmy · 2018-10-01T16:25:59Z

tagging with do not merge so that I don't merge it in the current "passed testing" state. @0xc0170 please remove the "do not merge" when you restart testing.

0xc0170 · 2018-10-02T09:24:12Z

/morph test

mbed-ci · 2018-10-02T11:41:40Z

Test : SUCCESS

Build number : 3011
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7917/3011

deepikabhavnani · 2018-10-02T15:06:43Z

We need to restart the tests (reports success but there is a hard fault in one of the tests), will restart

hey,curious to know how hardfault was discovered? I was not able to find and to be true didn;t go through logs of each and every test.

@theotherjimmy

Test was restarted and says success again, but would like to cross check for the same hardfault.

0xc0170 · 2018-10-02T19:13:41Z

/morph test

mbed-ci · 2018-10-03T03:54:42Z

Test : SUCCESS

Build number : 3019
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7917/3019

NirSonnenschein · 2018-10-03T07:55:09Z

still looks like a hard fault in LPC546XX,ARM (although tests report a pass):

logs:
21:09:40 [1538532634.70][CONN][RXD]
21:09:40 [1538532634.70][CONN][RXD] >>> Running case #18: 'test_remove_event_tail'...
21:09:40 [1538532634.80][CONN][RXD]
21:09:40 [1538532634.80][CONN][RXD] ++ MbedOS Fault Handler ++
21:09:40 [1538532634.80][CONN][RXD]
21:09:40 [1538532634.80][CONN][RXD] FaultType: HardFault
21:09:40 [1538532634.80][CONN][RXD]
21:09:40 [1538532634.80][CONN][RXD] Context:
21:09:40 [1538532634.80][CONN][INF] found KV pair in stream: {{__testcase_start;test_remove_event_tail}}, queued...
21:09:40 [1538532634.90][CONN][RXD] R0 : 00000007
21:09:40 [1538532634.90][CONN][RXD] R1 : 00000000
21:09:40 [1538532634.90][CONN][RXD] R2 : 00000007
21:09:40 [1538532634.90][CONN][RXD] R3 : 00000000
21:09:40 [1538532634.90][CONN][RXD] R4 : 00000000
21:09:40 [1538532634.90][CONN][RXD] R5 : 00000007
21:09:40 [1538532635.00][CONN][RXD] R6 : 00000007
21:09:40 [1538532635.00][CONN][RXD] R7 : 200025B8
21:09:40 [1538532635.00][CONN][RXD] R8 : 00000000
21:09:40 [1538532635.00][CONN][RXD] R9 : 20002548
21:09:40 [1538532635.00][CONN][RXD] R10 : 00000094
21:09:40 [1538532635.00][CONN][RXD] R11 : 20000F28
21:09:40 [1538532635.10][CONN][RXD] R12 : 00000000
21:09:40 [1538532635.10][CONN][RXD] SP : 200024F0
21:09:40 [1538532635.10][CONN][RXD] LR : 00006FFB
21:09:40 [1538532635.10][CONN][RXD] PC : FFF85780
21:09:40 [1538532635.10][CONN][RXD] xPSR : 41000000
21:09:40 [1538532635.10][CONN][RXD] PSP : 20002488
21:09:40 [1538532635.20][CONN][RXD] MSP : 20027FD8
21:09:40 [1538532635.20][CONN][RXD] CPUID: 410FC241
21:09:40 [1538532635.20][CONN][RXD] HFSR : 40000000
21:09:40 [1538532635.20][CONN][RXD] MMFSR: 00000001
21:09:40 [1538532635.20][CONN][RXD] BFSR : 00000000
21:09:40 [1538532635.20][CONN][RXD] UFSR : 00000000
21:09:40 [1538532635.30][CONN][RXD] DFSR : 00000008
21:09:40 [1538532635.30][CONN][RXD] AFSR : 00000000
21:09:40 [1538532635.30][CONN][RXD] Mode : Thread
21:09:40 [1538532635.30][CONN][RXD] Priv : Privileged
21:09:40 [1538532635.30][CONN][RXD] Stack: PSP
21:09:40 [1538532635.30][CONN][RXD]
21:09:40 [1538532635.40][CONN][RXD] -- MbedOS Fault Handler --
21:09:40 [1538532635.41][CONN][RXD]
21:09:40 [1538532635.41][CONN][RXD]
21:09:40 [1538532635.41][CONN][RXD]
21:09:40 [1538532635.41][CONN][RXD] ++ MbedOS Error Info ++
21:09:40 [1538532635.41][CONN][RXD] Error Status: 0x80FF013D Code: 317 Module: 255
21:09:40 [1538532635.50][CONN][RXD] Error Message: Fault exception
21:09:40 [1538532635.50][CONN][RXD] Location: 0xC8C9
21:09:40 [1538532635.50][CONN][RXD] Error Value: 0xFFF85780
21:09:41 [1538532635.60][CONN][RXD] Current Thread: Id: 0x20002688 Entry: 0xCAC1 StackSize: 0x1000 StackMem: 0x20001688 SP: 0x20027F90
21:09:41 [1538532635.71][CONN][RXD] For more info, visit: https://armmbed.github.io/mbedos-error/?error=0x80FF013D
21:09:41 [1538532635.71][CONN][RXD] -- MbedOS Error Info --

Heap statistics are used for analysing heap stats, but it doesn't tell anything about real heap usage or malloc overheads. Adding `overhead_size` element will help users to get the real heap usage.

deepikabhavnani · 2018-10-03T21:53:27Z

Issue might be related to ARMmbed/DAPLink#504.

Rebased and verified test locally, no error for LPC546xx Eval board with latest daplink (with above fix). The PR is merged but not released. Daplink firmware should be updated on LPC546xx devices in CI.

0xc0170 · 2018-10-04T15:22:37Z

/morph build

deepikabhavnani · 2018-10-04T18:41:44Z

/morph build

mbed-ci · 2018-10-04T19:39:31Z

Build : SUCCESS

Build number : 3236
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7917/

Triggering tests

/morph test
/morph mbed2-build

mbed-ci · 2018-10-05T05:18:29Z

Test : SUCCESS

Build number : 3043
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/7917/3043

0xc0170 · 2018-10-05T11:57:35Z

/morph export-build

mbed-ci · 2018-10-05T13:28:44Z

Exporter Build : SUCCESS

Build number : 2828
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/7917/

0xc0170 added the needs: review label Aug 29, 2018

0xc0170 requested a review from c1728p9 August 29, 2018 10:53

cmonr added the release-version: 5.10.0-rc1 label Aug 29, 2018

deepikabhavnani added needs: work and removed needs: review release-version: 5.10.0-rc1 labels Aug 31, 2018

deepikabhavnani force-pushed the heap_overhead branch from 7683164 to 2076168 Compare September 5, 2018 20:58

deepikabhavnani force-pushed the heap_overhead branch 2 times, most recently from bc105d5 to 70feb50 Compare September 5, 2018 21:31

deepikabhavnani added needs: review and removed needs: work labels Sep 5, 2018

deepikabhavnani requested a review from SeppoTakalo September 5, 2018 21:35

deepikabhavnani force-pushed the heap_overhead branch 2 times, most recently from bd9f02c to 2612aba Compare September 10, 2018 14:43

SeppoTakalo reviewed Sep 13, 2018

View reviewed changes

SeppoTakalo approved these changes Sep 14, 2018

View reviewed changes

c1728p9 approved these changes Sep 14, 2018

View reviewed changes

0xc0170 approved these changes Sep 21, 2018

View reviewed changes

0xc0170 added needs: CI and removed needs: review labels Sep 21, 2018

cmonr added ready for merge and removed needs: CI labels Sep 29, 2018

0xc0170 removed the ready for merge label Oct 1, 2018

theotherjimmy added needs: CI do not merge and removed needs: CI labels Oct 1, 2018

Add overhead_size element to get the overhead of stats

5f58828

Heap statistics are used for analysing heap stats, but it doesn't tell anything about real heap usage or malloc overheads. Adding `overhead_size` element will help users to get the real heap usage.

deepikabhavnani force-pushed the heap_overhead branch from 2612aba to 5f58828 Compare October 3, 2018 21:50

0xc0170 added release-version: 5.10.1 and removed do not merge release-version: 5.10.2 labels Oct 4, 2018

0xc0170 added ready for merge and removed needs: CI labels Oct 5, 2018

0xc0170 merged commit 8adb8c3 into ARMmbed:master Oct 5, 2018

0xc0170 removed the ready for merge label Oct 5, 2018

deepikabhavnani deleted the heap_overhead branch October 5, 2018 14:01

deepikabhavnani mentioned this pull request Oct 5, 2018

Heap stats are not reporting true values #7343

Closed

Add overhead count in heap stats #7917

Add overhead count in heap stats #7917

Uh oh!

Conversation

deepikabhavnani commented Aug 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Pull request type

Uh oh!

deepikabhavnani commented Aug 30, 2018

Uh oh!

deepikabhavnani commented Aug 31, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

deepikabhavnani commented Sep 5, 2018

Uh oh!

deepikabhavnani commented Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cmonr commented Sep 29, 2018

Uh oh!

mbed-ci commented Sep 29, 2018

Triggering tests

Uh oh!

mbed-ci commented Sep 29, 2018

Uh oh!

mbed-ci commented Sep 29, 2018

Uh oh!

0xc0170 commented Oct 1, 2018

Uh oh!

theotherjimmy commented Oct 1, 2018

Uh oh!

0xc0170 commented Oct 2, 2018

Uh oh!

mbed-ci commented Oct 2, 2018

Uh oh!

deepikabhavnani commented Oct 2, 2018

Uh oh!

0xc0170 commented Oct 2, 2018

Uh oh!

mbed-ci commented Oct 3, 2018

Uh oh!

NirSonnenschein commented Oct 3, 2018

Uh oh!

deepikabhavnani commented Oct 3, 2018

Uh oh!

0xc0170 commented Oct 4, 2018

Uh oh!

deepikabhavnani commented Oct 4, 2018

Uh oh!

mbed-ci commented Oct 4, 2018

Triggering tests

Uh oh!

mbed-ci commented Oct 5, 2018

Uh oh!

0xc0170 commented Oct 5, 2018

Uh oh!

mbed-ci commented Oct 5, 2018

Uh oh!

Uh oh!

deepikabhavnani commented Aug 28, 2018 •

edited

Loading

deepikabhavnani commented Aug 31, 2018 •

edited

Loading

deepikabhavnani commented Sep 12, 2018 •

edited

Loading