作者:Pika
链接:
https://juejin.cn/post/7432327923213205555
本文由作者授权发布。
最近都在更新鸿蒙相关的话题,android的文章比较少,我们来聊一个Android中有趣的话题,还是GC。
我们都知道Java虚拟机中都会有垃圾回收机制(GC),有了垃圾回收机制的存在,虚拟机可以根据策略去回收一些被释放的Java对象,从而保证整个内存的空间不至于无限增长,一般的策略就是看Java虚拟机中的内存空间大小占比去决定要不要进行GC,那么问题来了,Native中分配的内存,会不会影响虚拟机的GC回收策略呢?换句话来说,Native内存越多,会不会触发GC?答案是,会的,ART虚拟机中有这种机制,不过可以说是“间接的”。
在ART GC策略中,触发常规GC时,GC大致触发可以分为策略触发GC RequestConcurrentGC 与分配时GC(AllocateInternalWithGc),前者通常是内存总数达到一定策略时触发,后者是内存不足时进行的GC触发,GC的原因以GcCause这个枚举类给出,而我们今天的话题就是,GcCause为kGcCauseForNativeAlloc时因为Native 内存分配引起的GC。
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;l=3989
enum GcCause {
...
// GC triggered for a native allocation when NativeAllocationGcWatermark is exceeded.
// (This may be a blocking GC depending on whether we run a non-concurrent collector).
kGcCauseForNativeAlloc,
那么什么时候会触发GcCause为kGcCauseForNativeAlloc类型的回收呢,其实它的链路非常简单,通过CheckGCForNative函数触发,GC的策略依据是否可以并发有些小区别(is_gc_concurrent 是否为true),关键在于这个变量gc_urgency 是否大于等于1,如果满足条件则才会进行gc处理。
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=a4b3ffea3814d5d82b1aac9969b492d3642dbce4;bpv=1;bpt=1;l=4236?q=CheckGCForNative
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=a4b3ffea3814d5d82b1aac9969b492d3642dbce4;bpv=1;bpt=1;l=4239?q=CheckGCForNative
inline void Heap::CheckGCForNative(Thread* self) {
bool is_gc_concurrent = IsGcConcurrent();
uint32_t starting_gc_num = GetCurrentGcNum();
size_t current_native_bytes = GetNativeBytes();
float gc_urgency = NativeMemoryOverTarget(current_native_bytes, is_gc_concurrent);
// 关键在于gc_curgency 这个系数是否大于等于1,小于则不触发gc
if (UNLIKELY(gc_urgency >= 1.0)) {
根据是否支持并行gc有小区别
if (is_gc_concurrent) {
bool requested =
RequestConcurrentGC(self, kGcCauseForNativeAlloc, /*force_full=*/true, starting_gc_num);
if (requested && gc_urgency > kStopForNativeFactor
&& current_native_bytes > stop_for_native_allocs_) {
// We're in danger of running out of memory due to rampant native allocation.
if (VLOG_IS_ON(heap) || VLOG_IS_ON(startup)) {
LOG(INFO) <"Stopping for native allocation, urgency: " < }
// Count how many times we do this, so we can warn if this becomes excessive.
// Stop after a while, out of excessive caution.
static constexpr int kGcWaitIters = 20;
for (int i = 1; i <= kGcWaitIters; ++i) {
if (!GCNumberLt(GetCurrentGcNum(), max_gc_requested_.load(std::memory_order_relaxed))
|| WaitForGcToComplete(kGcCauseForNativeAlloc, self) != collector::kGcTypeNone) {
break;
}
CHECK(GCNumberLt(starting_gc_num, max_gc_requested_.load(std::memory_order_relaxed)));
if (i % 10 == 0) {
LOG(WARNING) <"Slept " <" times in native allocation, waiting for GC";
}
static constexpr int kGcWaitSleepMicros = 2000;
usleep(kGcWaitSleepMicros); // Encourage our requested GC to start.
}
}
} else {
CollectGarbageInternal(NonStickyGcType(), kGcCauseForNativeAlloc, false, starting_gc_num + 1);
}
}
}
gc_urgency 这个变量其实就是当前的权重,因为触发GC是有成本的,过度使用并不好,我们可以看到这个系数通过NativeMemoryOverTarget 方法计算,参数是 GetNativeBytes() 得到的变量,即当前Native占用内存大小。
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=a4b3ffea3814d5d82b1aac9969b492d3642dbce4;bpv=1;bpt=1;l=4208?q=CheckGCForNativehttps://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=a4b3ffea3814d5d82b1aac9969b492d3642dbce4;bpv=1;bpt=1;l=2723?q=CheckGCForNative
inline float Heap::NativeMemoryOverTarget(size_t current_native_bytes, bool is_gc_concurrent) {
判断上一次触发CheckGCForNative 的内存大小,如果不超过上一次,那么不触发,这是一个优化策略,即然上次都能够分配成功了,那么也就不必要再触发一次gc
size_t old_native_bytes = old_native_bytes_allocated_.load(std::memory_order_relaxed);
if (old_native_bytes > current_native_bytes) {
记录本次数值,结束
old_native_bytes_allocated_.store(current_native_bytes, std::memory_order_relaxed);
return 0.0;
} else {
否则通过当前已经使用的native内存与上一次记录的内存进行计算
size_t new_native_bytes = UnsignedDifference(current_native_bytes, old_native_bytes);
size_t weighted_native_bytes = new_native_bytes / kNewNativeDiscountFactor
+ old_native_bytes / kOldNativeDiscountFactor;
size_t add_bytes_allowed = static_cast<size_t>(
NativeAllocationGcWatermark() * HeapGrowthMultiplier());
size_t java_gc_start_bytes = is_gc_concurrent
? concurrent_start_bytes_
: target_footprint_.load(std::memory_order_relaxed);
size_t adj_start_bytes = UnsignedSum(java_gc_start_bytes,
add_bytes_allowed / kNewNativeDiscountFactor);
最终的结果是通过native内存的比值与java内存的系数做运算得出最终的比值
return static_cast<float>(GetBytesAllocated() + weighted_native_bytes)
/ static_cast<float>(adj_start_bytes);
}
}
NativeMemoryOverTarget 函数通过Native内存与上一次记录的内存进行了加权计算,和最终Java内存使用得到最终的数值,这其实是一个内存调控算法,目的就是为了在尽力减少GC的情况下,保证最大效率触发GC。我们再来学习一下ART虚拟机是如何统计Native内存的。size_t Heap::GetNativeBytes() {
size_t malloc_bytes;
#if defined(__BIONIC__) || defined(__GLIBC__) || defined(ANDROID_HOST_MUSL)
IF_GLIBC(size_t mmapped_bytes;)
struct mallinfo mi = mallinfo();
// In spite of the documentation, the jemalloc version of this call seems to do what we want,
// and it is thread-safe.
if (sizeof(size_t) > sizeof(mi.uordblks) && sizeof(size_t) > sizeof(mi.hblkhd)) {
// Shouldn't happen, but glibc declares uordblks as int.
// Avoiding sign extension gets us correct behavior for another 2 GB.
malloc_bytes = (unsigned int)mi.uordblks;
IF_GLIBC(mmapped_bytes = (unsigned int)mi.hblkhd;)
} else {
malloc_bytes = mi.uordblks;
IF_GLIBC(mmapped_bytes = mi.hblkhd;)
}
// From the spec, it appeared mmapped_bytes <= malloc_bytes. Reality was sometimes
// dramatically different. (b/119580449 was an early bug.) If so, we try to fudge it.
// However, malloc implementations seem to interpret hblkhd differently, namely as
// mapped blocks backing the entire heap (e.g. jemalloc) vs. large objects directly
// allocated via mmap (e.g. glibc). Thus we now only do this for glibc, where it
// previously helped, and which appears to use a reading of the spec compatible
// with our adjustment.
#if defined(__GLIBC__)
if (mmapped_bytes > malloc_bytes) {
malloc_bytes = mmapped_bytes;
}
#endif // GLIBC
#else // Neither Bionic nor Glibc
// We should hit this case only in contexts in which GC triggering is not critical. Effectively
// disable GC triggering based on malloc().
malloc_bytes = 1000;
#endif
return malloc_bytes + native_bytes_registered_.load(std::memory_order_relaxed);
// An alternative would be to get RSS from /proc/self/statm. Empirically, that's no
// more expensive, and it would allow us to count memory allocated by means other than malloc.
// However it would change as pages are unmapped and remapped due to memory pressure, among
// other things. It seems risky to trigger GCs as a result of such changes.
}
我们可以看到,最终的结果malloc_bytes是通过mallinfo这个系统调用返回的,mallinfo是一个用于获取内存分配信息的函数。它提供了有关堆内存(heap memory)的详细使用情况,包括已分配的内存块数量、空闲的内存块数量、内存碎片等信息。这个函数返回一个struct mallinfo类型的结构体,附上链接 ,这里面记录着内存的大多数信息,我们只拿ART用到的uordblks ,与hblkhd 解释一下:
https://man7.org/linux/man-pages/man3/mallinfo.3.htmlhttps://cs.android.com/android/platform/superproject/main/+/main:prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.17-4.8/sysroot/usr/include/malloc.h;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=110
https://cs.android.com/android/platform/superproject/main/+/main:prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.17-4.8/sysroot/usr/include/malloc.h;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=107- 代表普通(非 mmap 分配的)空闲内存块的数量。这些空闲块可以在后续的内存分配请求中被使用。例如,当程序释放了一些之前通过malloc分配的内存时,这些内存块可能会被归类为普通空闲内存块,ordblks的值就会相应地增加。
- 用于记录程序请求使用mmap分配大块内存(以MMAP_THRESHOLD为界,这个阈值可以通过系统参数等方式设置)的次数。当程序需要分配较大的内存块时,可能会使用mmap方式,每请求一次,hblks的值就会增加。
这两个变量其实就是记录中我们在Native中常用的内存分配函数malloc系列(calloc等)与mmap函数所分配的内存数量。最终返回的结果是malloc_bytes(mmapped_bytes > malloc_bytes 取两种中大的) + native_bytes_registered ,那么这个native_bytes_registered 又是个啥,这跟我们下文会讲到的NativeAllocationRegistry 有关,我们看下它的赋值,其实就是通过RegisterNativeFree函数的bytes参数得到的。
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=2724
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.cc;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=2726
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.h;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=1517
void Heap::RegisterNativeFree(JNIEnv*, size_t bytes) {
size_t allocated;
size_t new_freed_bytes;
do {
allocated = native_bytes_registered_.load(std::memory_order_relaxed);
new_freed_bytes = std::min(allocated, bytes);
// We should not be registering more free than allocated bytes.
// But correctly keep going in non-debug builds.
DCHECK_EQ(new_freed_bytes, bytes);
} while (!native_bytes_registered_.CompareAndSetWeakRelaxed(allocated,
allocated - new_freed_bytes));
}
最终通过两者计算得出阈值,从而触发是否要产生GC,那么CheckGCForNative会在哪里被触发呢?接下来看。
为什么ART要根据Native内存占用触发GC呢?大家可能会问,为啥GC要判断Native内存大小触发呢?GC回收的不是Java虚拟机的内存吗,跟Native内存有啥关系,Native内存不是由使用者自己回收吗?其实这是有原因的。在日常JNI开发中,我们通常都需要用到Java对象与Native对象,通过Java对象保存Native对象的指针是一个很常见的操作,比如Bitmap。public final class Bitmap {
private final long mNativePtr;
....
外部通过Java对象去管理,内部比如内存的分配放到Native内存中。我们知道Java对象可以由虚拟机去回收,那么Java对象持有的Native对象怎么办呢?在最早期的版本,开发者通过在finalize 函数中主动调用释放所持有的Native内存,比如Camera2中比较典型的CameraMetaDataNative,其实就是利用finalize方法进行最后的Native对象内存回收。 @Override
protected void finalize() throws Throwable {
try {
close();
} finally {
super.finalize();
}
}
private void close() {
// Delete native pointer, but does not clear it
nativeClose(mMetadataPtr);
mMetadataPtr = 0;
if (mBufferSize > 0) {
VMRuntime.getRuntime().registerNativeFree(mBufferSize);
}
mBufferSize = 0;
}
看似很合理对吧,但是我们也同样知道,一个垃圾对象的finalize方法其实是依赖GC的执行的,只有GC后才会被调用finalize方法,然而在早期的android系统中,GC策略只会依据Java内存大小的容量进行,很有可能出现的情况是,即便一个对象处于无用状态,但是因为还没有达到Java内存的阈值,因此也不会进行GC,这就导致了其所持有的Native内存一直也是属于不释放的状态,造成无用的Native内存过多的问题。这类问题我搜了一下,发现还真的存在,比如Android Camera 内存问题剖析。
https://juejin.cn/post/6862508868438589447#heading-6
同时系统大部分关键类也采取了上述的设计,因此官方在后续引入了一个叫NativeAllocationRegistry 的类,用于管理Java对象以及其所持有的Native内存,当用其管理内存时,会调用registerNativeAllocation方法,此时就会通过JNI方法(VMRuntime_registerNativeAllocation等)调用到我们上面说的CheckGCForNative方法。
https://cs.android.com/android/platform/superproject/main/+/main:libcore/luni/src/main/java/libcore/util/NativeAllocationRegistry.java;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=554?q=NativeAllocationRegistry&gsn=registerNativeAllocation&gs=KYTHE%3A%2F%2Fkythe%3A%2F%2Fandroid.googlesource.com%2Fplatform%2Fsuperproject%2Fmain%2F%2Fmain%3Flang%3Djava%3Fpath%3Dlibcore.util.NativeAllocationRegistry%23b93ed1e878cc5ad35fc81568870f6ea8257b4471ca9bee5a93331784b16a3cff
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/native/dalvik_system_VMRuntime.cc;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=290?q=CheckGCForNative&gsn=VMRuntime_registerNativeAllocation&gs=KYTHE%3A%2F%2Fkythe%3A%2F%2Fandroid.googlesource.com%2Fplatform%2Fsuperproject%2Fmain%2F%2Fmain%3Flang%3Dc%252B%252B%3Fpath%3Dart%2Fruntime%2Fnative%2Fdalvik_system_VMRuntime.cc%2310ELAHEa8LH3ZRw0MivTrvXGXvBh6a_kuFCKa47oSYI
private static void registerNativeAllocation(long size) {
VMRuntime runtime = VMRuntime.getRuntime();
if ((size & IS_MALLOCED) != 0) {
final long notifyImmediateThreshold = 300000;
if (size >= notifyImmediateThreshold) {
runtime.notifyNativeAllocationsInternal();
} else {
runtime.notifyNativeAllocation();
}
} else {
runtime.registerNativeAllocation(size);
}
}
从而达到Native内存阈值检测的目的,即使触发一次GC,从而让依赖GC回收的对象尽快回收,从而让其持有的Native对象内存也一并被回收。这是一个非常大的改进,后续很多Android系统核心类都是用了这个策略,从而保证了Native内存的有效回收。我们刚刚上文还提到一个native_bytes_registered 对象,它其实就是通过NativeAllocationRegistry传入的Native对象大小,因为NativeAllocationRegistry创建时可以让使用者提供它当前所持有的Native对象的大小,通过registerNativeFree方法,这样就不会错过使用者告知的Native内存,使得整个内存统计更加准确。
https://cs.android.com/android/platform/superproject/main/+/main:art/runtime/gc/heap.h;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=1517
https://cs.android.com/android/platform/superproject/main/+/main:libcore/luni/src/main/java/libcore/util/NativeAllocationRegistry.java;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=569?q=registerNativeFree&ss=android%2Fplatform%2Fsuperproject%2Fmain&gsn=registerNativeFree&gs=KYTHE%3A%2F%2Fkythe%3A%2F%2Fandroid.googlesource.com%2Fplatform%2Fsuperproject%2Fmain%2F%2Fmain%3Flang%3Djava%3Fpath%3Dlibcore.util.NativeAllocationRegistry%239496ea8d6c9d8d521832274729090f8d2940d819e8d45951e80aa109eebe2b5e
// Inform the garbage collector of deallocation, if appropriate.
private static void registerNativeFree(long size) {
if ((size & IS_MALLOCED) == 0) {
VMRuntime.getRuntime().registerNativeFree(size);
}
}
值得一提的是,我们会在代码中看到IS_MALLOCED 这个标记,这个也是创建NativeAllocationRegistry时由使用者告知的,如果它是通过malloc所分配的,其size属性就会在最后一位加上,IS_MALLOCED 这个标记。https://cs.android.com/android/platform/superproject/main/+/main:libcore/luni/src/main/java/libcore/util/NativeAllocationRegistry.java;drc=517003e1168f7d0e07cac9c60e67b73f0e28dbde;bpv=1;bpt=1;l=81?q=registerNativeFree&ss=android%2Fplatform%2Fsuperproject%2Fmain private NativeAllocationRegistry(@NonNull ClassLoader classLoader, @NonNull Class clazz,
long freeFunction, long size, boolean mallocAllocation) {
if (size 0) {
throw new IllegalArgumentException("Invalid native allocation size: " + size);
}
this.clazz = Objects.requireNonNull(clazz);
this.classLoader = Objects.requireNonNull(classLoader);
this.freeFunction = freeFunction;
this.size = mallocAllocation ? (size | IS_MALLOCED) : (size & ~IS_MALLOCED);
synchronized(NativeAllocationRegistry.class) {
registries.put(this, null);
}
}
我们可以验证一下是不是Native内存过多时NativeAllocationRegistry是否按照正常的策略回收,我们可以通过inline hook RequestConcurrentGC这个方法,去查看其GcCause是不是kGcCauseForNativeAlloc 去验证我们整个流程,对应的符号是_ZN3art2gc4Heap19RequestConcurrentGCEPNS_6ThreadENS0_7GcCauseEbj。 值得注意RequestConcurrentGC是Heap类的成员方法,因此其函数类型需要多加上void *heap这个this指针,不要忘记噢。bool hookRequestConcurrentGC(void *heap, void *self,
enum GcCause cause,
bool force_full,
uint32_t observed_gc_num) {
__android_log_print(ANDROID_LOG_ERROR, "hello", "gc type %d", cause);
bool result = ((gc_temp) heap_record_gc)(heap, self, cause, force_full, observed_gc_num);
__android_log_print(ANDROID_LOG_ERROR, "hello", "result ", cause);
return result;
}
当我们分配一个大型malloc对象时,就会看到由系统的NativeAllocationRegistry触发的gc,其type正是kGcCauseForNativeAlloc。值得注意的是,我们的创建的malloc对象并不会被回收,因为这个对象是我们自己管理的,会被回收的内存只是由无用Java对象所持有的Native内存。通过上面我们了解到,在Native内存过多时触发一次GC是有好处的,能够加快一些无用的Java类回收,也能够保证其所持有的Native内存回收,因此在一些早期的Android版本之上或者是没有用到NativeAllocationRegistry进行管理的Java类中,我们是可以通过检测当前Native内存的大小,主动触发GC的操作去提高内存回收的效率。
通过学习GC类型为kGcCauseForNativeAlloc的整个链路,能够帮助我们更加全面的认识ART的GC细节,同时其在高版本所用到的回收策略,也是可以被我们运用在低版本中,达到性能优化的目的。
最后推荐一下我做的网站,玩Android: wanandroid.com ,包含详尽的知识体系、好用的工具,还有本公众号文章合集,欢迎体验和收藏!
推荐阅读:
扫一扫 关注我的公众号
如果你想要跟大家分享你的文章,欢迎投稿~
┏(^0^)┛明天见!