I just discovered what appears to be working weak-modules support for Lustre 2.5.1 client modules on RHEL6. I saw our lustre filesystem was mounted on a host running 2.6.32-431.29.2.el6.x86_64 kernel but with client modules compiled for 2.6.32_431.23.3.el6.x86_64. Sure enough, the symlinks are in place under /lib/modules/<kernel-version>. I tried booting into a couple of other kernel versions with module symlinks and the lustre client worked there too. This is a pretty significant feature......when was it introduced? Is it supported? -------- Yemi On Sep 15, 2011, at 4:22 PM, Adeyemi Adesanya wrote: > > Hi Brian. > > I don't even see compatibility between "-274" kernels. I built and installed on 2.6.18-274.3.1.el5 but the only module that got symlinked under 2.6.18-274.el5 was libcfs.ko. > Thanks for the info regarding RedHat and kABI. > > ------ > Yemi > > On Sep 15, 2011, at 4:12 PM, Brian J. Murrell wrote: > >> On 11-09-15 06:57 PM, Adesanya, Adeyemi wrote: >>> >>> I just dug up a message from lustre-discuss last year regarding support for weak-modules. It would be great I didn't have to rebuild the lustre-modules client RPM (lustre 1.8.6) against every new RHEL5 kernel that gets released. >> >> Indeed, it would be good. You don't have this problem for SLES kernels, >> FWIW. But for RH kernels, weak (Lustre at least) modules are not >> possible due to RedHat only supporting a subset of the kABI for weak >> modules and Lustre utilizes symbols outside of that subset (they call >> the "whitelist"). >> >> I tried to get the whitelist updated for Lustre quite a while ago but >> was met with silence. >> >>> weak-modules reported that nearly all of the modules were incompatible with other kernels including the recent 2.6.18-274.el5, 2.6.18-238.19.1.el5, etc. >> >> I'm not positive but I don't think those two kernels are intended to be >> ABI compatible. My (rather old, so not great) understanding is that the >> first component after the 2.6.18- identifies kernels that are supposed >> to be binary compatible and need to be the same between two kernels to >> ensure kABI compatibility. >> >> Cheers, >> b. >> >> -- >> Brian J. Murrell >> Senior Software Engineer >> Whamcloud, Inc. >> > _______________________________________________ HPDD-discuss mailing list HPDD-discuss(a)lists.01.org https://lists.01.org/mailman/listinfo/hpdd-discuss

The weak-updates bits added to the spec some time ago partially work, but the post install and uninstall scripts are incomplete. If you are interested in this feature, you might want look at LU-5614. I explored getting this feature into mainline client builds properly : using the %kernel_module_package macro to pick up vendor defined install scripts, instead of trying to maintain the scripts in the spec. The essential problem for %kernel_module_package is that the implementations can be confused by multiple versions of the kernel being installed in the build environment. With SGI's build environment, I was guaranteed to only have the enumerated flavors of a single kernel release in my build environment. I do not have control over this in other environments, and I have verified that Intel's jenkins environment has multiple kernel releases installed. The mechanisms for overcoming this - specifying the kernel released to build for - are not standardized among the distributions. The spec changes are intrusive enough that we probably want to break off another spec for use when a weak-modules build is requested by configure. It's an annoying problem, but not an especially difficult one. I have been thinking that dkms may actually be a better solution for my customers, so I've been looking in that direction when I do have time to consider this problem. As for kabi compatibility : I did find compatibility problems with the RHEL 2.6.32-YYY-stuff update releases not being compatible with previous 2.6.32-YYY releases. Rare, but it did happen. From my experience, each YYY level is generally associated with an RH dot level release. Compatibility is checked for all of the symbols in the list for all of the kernels in a major release stream. They are much more aggressive about breaking unlisted symbols between dot releases, but it occasionally happens within a series of dot release updates. We submitted a symbol list to RH a couple of times, but many symbols Lustre uses were not accepted, and for good reasons. SuSE unwisely picked up all of the kabi symbols we submitted to them, so I would not expect Lustre to have a problem with kABI compatibility within a SLES release. Adesanya, Adeyemi wrote: > Giving this email a bump for Andreas. We spoke at SC14 this afternoon. > > ------- > Yemi > > On Nov 12, 2014, at 7:24 PM, Adesanya, Adeyemi <yemi(a)slac.stanford.edu&gt; wrote: > >> I just discovered what appears to be working weak-modules support for Lustre 2.5.1 client modules on RHEL6. I saw our lustre filesystem was mounted on a host running 2.6.32-431.29.2.el6.x86_64 kernel but with client modules compiled for 2.6.32_431.23.3.el6.x86_64. Sure enough, the symlinks are in place under /lib/modules/<kernel-version>. I tried booting into a couple of other kernel versions with module symlinks and the lustre client worked there too. This is a pretty significant feature......when was it introduced? Is it supported? >> >> -------- >> Yemi >> >> On Sep 15, 2011, at 4:22 PM, Adeyemi Adesanya wrote: >> >>> Hi Brian. >>> >>> I don't even see compatibility between "-274" kernels. I built and installed on 2.6.18-274.3.1.el5 but the only module that got symlinked under 2.6.18-274.el5 was libcfs.ko. >>> Thanks for the info regarding RedHat and kABI. >>> >>> ------ >>> Yemi >>> >>> On Sep 15, 2011, at 4:12 PM, Brian J. Murrell wrote: >>> >>>> On 11-09-15 06:57 PM, Adesanya, Adeyemi wrote: >>>>> I just dug up a message from lustre-discuss last year regarding support for weak-modules. It would be great I didn't have to rebuild the lustre-modules client RPM (lustre 1.8.6) against every new RHEL5 kernel that gets released. >>>> Indeed, it would be good. You don't have this problem for SLES kernels, >>>> FWIW. But for RH kernels, weak (Lustre at least) modules are not >>>> possible due to RedHat only supporting a subset of the kABI for weak >>>> modules and Lustre utilizes symbols outside of that subset (they call >>>> the "whitelist"). >>>> >>>> I tried to get the whitelist updated for Lustre quite a while ago but >>>> was met with silence. >>>> >>>>> weak-modules reported that nearly all of the modules were incompatible with other kernels including the recent 2.6.18-274.el5, 2.6.18-238.19.1.el5, etc. >>>> I'm not positive but I don't think those two kernels are intended to be >>>> ABI compatible. My (rather old, so not great) understanding is that the >>>> first component after the 2.6.18- identifies kernels that are supposed >>>> to be binary compatible and need to be the same between two kernels to >>>> ensure kABI compatibility. >>>> >>>> Cheers, >>>> b. >>>> >>>> -- >>>> Brian J. Murrell >>>> Senior Software Engineer >>>> Whamcloud, Inc. >>>> >> _______________________________________________ >> HPDD-discuss mailing list >> HPDD-discuss(a)lists.01.org >> https://lists.01.org/mailman/listinfo/hpdd-discuss > _______________________________________________ > HPDD-discuss mailing list > HPDD-discuss(a)lists.01.org > https://lists.01.org/mailman/listinfo/hpdd-discuss

Adesanya, Adeyemi wrote: > Thanks for responding. Can I trust the module symlinks that are generated when new RHEL6 kernels are installed? I can deal with too-aggressive symbol checking that breaks compatibility between releases but my fear is that symlinks get generated for modules that are not compatible. The module tools check symbol compatibility for all symbols, not just the whitelisted ones - so any changes should be detected, and no link will be created for a module which references changed symbols. There are limitations. The scripts will install links for all enumerated modules for which it can resolve all of the symbols. If symbols do not resolve, you can end up with some of the modules from the package being linked, while others are not. This can be annoying, but is easily identified and resolved. Not all kABI changes are detected by the computed symbol versions. It is a very good and improving mechanism, but is inherently incomplete. A change in lock ordering, for example, can easily escape detection. My favorite undetected change in the kABI was the kernel's page size. I believe it is tracked now.