Order in which linux device drivers are probed!

The oder in which the device driver are initialized depends on the level of initcall in the driver. The levels range from 0-7.

#define pure_initcall(fn) __define_initcall("0",fn,0)

#define core_initcall(fn) __define_initcall("1",fn,1)

#define core_initcall_sync(fn) __define_initcall("1s",fn,1s)

#define postcore_initcall(fn) __define_initcall("2",fn,2)

#define postcore_initcall_sync(fn) __define_initcall("2s",fn,2s)

#define arch_initcall(fn) __define_initcall("3",fn,3)

#define arch_initcall_sync(fn) __define_initcall("3s",fn,3s)

#define subsys_initcall(fn) __define_initcall("4",fn,4)

#define subsys_initcall_sync(fn) __define_initcall("4s",fn,4s)

#define fs_initcall(fn) __define_initcall("5",fn,5)

#define fs_initcall_sync(fn) __define_initcall("5s",fn,5s)

#define rootfs_initcall(fn) __define_initcall("rootfs",fn,rootfs)

#define device_initcall(fn) __define_initcall("6",fn,6)

#define device_initcall_sync(fn) __define_initcall("6s",fn,6s)

#define late_initcall(fn) __define_initcall("7",fn,7)

#define late_initcall_sync(fn) __define_initcall("7s",fn,7s)

Ref: include/linux/init.h

Below is the issue that I faced.

I have an mfd device and the count of the clients goes upto 14. Now that I am interested in only 3 among them(say x, y and z) and the order that I want them to be probed in is x->y->z.

In my mfd core driver, the order in which I add the devices also suits my requirement, i.e x->y->z.

The level of initcall in all the 3 drivers are same, say subsys_initcall_sync(). Also all the 3 drivers(x, y and z) are in the same folder.

The expected result would be 'x' should be probed first followed by 'y' and 'z'.

But no, the order that I got is x->z->y.

I tried altering the order in which the mfd devices are added but no use. The actual problem was in the Makefile, the order in which linking had happened.

In makefile

obj-$(CONFIG_XXXX) += x.0 z.0 y.0

Here is where the issue was, later I changed the order in makefile and things started working as expected.

So remember the order in which linking happens also matters!

Notifiers in Linux kernel

Adding new notifiers and calling the notifier call chain.
Assume that are two drivers and one driver will have to notify the other.
Driver-1
/* atomic notifier initialization */
ATOMIC_NOTIFIER_HEAD(simple_notifier_list);

unsigned long val;
struct device_info *di;
/* later in some part of the driver call the notifier call chain */
atomic_notifier_call_chain(&simple_notifier_list, val, (void *)di);
Driver-2
static int notify_call_handler(notifier_block *n, unsigned long event, void *data)
{
}
static struct notifier_block simple_nb = {
.notifier_call = simple_notify_handler;
}
atomic_notifier_chain_register(&simple_notifier_list, &simple_nb)

iotop: Per Process I/O Usage | Linux Magazine

Monitoring the IO usage of individual processes on a system has been much easier with iotop. It also includes some interesting and very useful performance monitoring features.

iotop: Per Process I/O Usage | Linux Magazine

Receive Packet Steering (RPS) on Linux 2.6.35

This patch implements software receive side packet steering (RPS). RPS distributes the load of received packet processing across multiple CPUs. Problem statement: Protocol processing done in the NAPI context for received packets is serialized per device queue and becomes a bottleneck under high packet load. This substantially limits pps that can be achieved on a single queue NIC and provides no scaling with multiple cores. (lwn.net: Software receive packet steering)

If you want to find out whether RPS is working, you have to look at /proc/softirqs instead (eg. with watch -n1 cat /proc/softirqs):

                CPU0       CPU1

       HI:          0          0

    TIMER:  480622794  476948579

   NET_TX:   25311134   27075847     <-----

   NET_RX: 1388399338 4191697027     <-----

    BLOCK:    4632803          3

 BLOCK_IOPOLL:          0          0

  TASKLET:         21          4

    SCHED:  154913375  158601463

  HRTIMER:    1576760    2361409

      RCU:  421549961  407634645 

Enabling RPS specifically for a particular interface:

# cat /sys/class/net/eth0/queues/rx-0/rps_cpus 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000, 00000000,00000000,00000000,00000000,00000000,00000000,00000000  # echo f > /sys/class/net/eth0/queues/rx-0/rps_cpus # cat /sys/class/net/eth0/queues/rx-0/rps_cpus 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000, 00000000,00000000,00000000,00000000,00000000,00000000,0000000f

Concept of Cache Line

cache line - The smallest unit of memory than can be transferred between the main memory and the L1/L2 cache.

Rather than reading a single word or byte from main memory at a time, each cache entry is usually holds a certain number of words, known as a "cache line" or "cache block" and a whole line is read and cached at once. This takes advantage of the principle of locality of reference: if one location is read then nearby locations (particularly following locations) are likely to be read soon afterward. It can also take advantage of page-mode DRAM which allows faster access to consecutive locations.

The cache line is generally fixed in size, typically ranging from 16 to 256 bytes. The effectiveness of the line size depends on the application, and cache circuits may be configurable to a different line size by the system designer. There are also numerous algorithms for dynamically adjusting line size in real time.