cmake: how to add a compiler flag according to current compiler version

I have a c library to maintain, which is built by cmake. When I start to build it, it tells me that

cc1: warning: unrecognized command line option "-Wno-incompatible-pointer-types" [enabled by default]

yes, my gcc version is 4.8.5 and does not have that flag. So I want to disable it when the compiler does not support that flag.

After some searching, I came to this solution

include(CheckCCompilerFlag)
check_c_compiler_flag(-Wno-invalid-offsetof HAS_NO_INVALID_OFFSETOF)
if (HAS_NO_INVALID_OFFSETOF)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-invalid-offsetof")
endif()

but when I add it to my CMakeLists.txt, it does not work, the above warning still comes out.

After searching the cmake documentation, it tells me that

A positive result from this check indicates only that the compiler did not issue a diagnostic message when given the flag. Whether the flag has any effect or even a specific one is beyond the scope of this module.

According to that, my gcc does not support that flag, it won't issue a diagnostic message, so cmake thinks that my gcc supports that flag?

Is there any way to make it act as the way I like it to do ?