Video for Linux Two - Driver Writer's Guide

Introduction

Why have this document when there is an API spec that covers all the functions? Because the API spec only covers the user-mode calls, and there are aspects of V4L2 that are only relevant to kernel-mode code. The purpose of this document is to describe how V4L2 driver modules hook into the system, document the V4L2 kernel-mode API, discuss driver implementation issues, and provide any other additional documentation applicable to driver writers.

This document assumes the reader is familiar with the V4L2 user-mode API spec, and the relevant topics of Linux device driver programming. Linux Device Drivers by Allessandro Rubini, ISBN 1-56592-292-1 seems to be a standard reference.
 
 

V4L2 Drivers & videodev

V4L2 is a two-layer driver system. The top layer is the videodev module. When videodev initializes, it registers as char major device 81, and registers its set of char driver method functions. All V4L2 drivers are really clients of videodev, and videodev calls the client drivers through V4L2 driver method functions. When a V4L2 driver initializes, it registers each device it will handle with videodev by passing to videodev a structure which contains the V4L2 driver methods, a minor device number, and a couple other details. The V4L2 methods are very similar to the regular Linux char driver methods, but have different parameters specially for V4L2 drivers. When an application makes a driver call, control passes first to the videodev method, which translates the file and/or inode structure pointers into the corresponding V4L2 structure pointer, and calls the V4L2 driver's method. Thus videodev acts as a thin shell around V4L2 drivers. Videodev is implemented as a module, and all V4L2 drivers are also modules.

The videodev module also includes a set of helper functions which V4L2 driver writers may find useful. For example, some simple queue management, translating a struct file * to the corresponding V4L2 structure, memory mapping helper functions and so on.
 
 

Driver Registration and Method Calls

When a driver initializes, it enumerates the devices in the system which it is going to handle, and for each one it fills in a separate struct v4l2_device structure and passes a pointer to the structure to v4l2_register_device() . V4l2_register_device() will make sure the minor number is available, and call the initialize method of the struct v4l2_device. Initialize is passed the device pointer that was passed to v4l2_register_device() . If initialize succeeds then the registration is complete. Videodev will store the pointer in an internal table, and call the method functions as the driver is accessed by applications.

Before calling v4l2_register_device() the driver must fill in the name, type, and minor fields, and the open method. The other methods and fields are optional. The type field uses the same values as the struct v4l2_capability.type field, which are the V4L2_TYPE_* symbols. It is possible to change method fields after registration. When the driver unloads, it calls v4l2_unregister_device() with the device pointer.
 

Design Note: Typically, drivers will internally define a much larger device structure that begins with a struct v4l2_device, followed by all the driver and device state information associated with the particular device. A pointer to this larger structure can be used (with appropriate casts) everywhere a struct v4l2_device * is called for.

Minor Device Numbers

For V4L2 devices, the minor numbers have to be given as module parameters. Drivers have to declare the parameter variables and export them. The parameter for video capture devices is unit_video, for codecs it is unit_codec, and so on. Fill in the minor field of each struct v4l2_device, and call v4l2_register_device(). The registration will fail if the minor is already in use.

static int unit_video[MAX_DEVICES] = {0, 1, 2, 3, };
MODULE_PARM(unit_video, "1-"__MODULE_STRING(MAX_DEVICES)"i");
...
mydevice[i].minor = unit_video[i];
if (v4l2_register_device(&mydevice[i]))
{ /* handle error */
}

V4L2 Driver Method Calls

The following is a list of the V4L2 driver method calls, each with a brief description and any relevant notes.

Method: int initialize(struct v4l2_device *v)

Called during registration if there is no error in v4l2_register_device() . The driver can do any remaining initialization that is required before open is called. Return a negative number for error. An error cancels the registration.

Method: int open(struct v4l2_device *v, int flags, void **idptr)

Called when a file descriptor is opened on the device file. Open is passed the device pointer that was used to register the device. The flags parameter is the flags passed to open() by the application. V4L2 devices can be opened more than once simultaneously, either by the same application, or by more than one application. The idptr parameter is a pointer to an ID value which will identify this particular open (like the private_data field in struct file). After open, all the other method calls are passed this id value. The driver needs to set *idptr to an id value for this open. The value can be anything, except zero (NULL) is not valid. The data type of (*idptr) is (void *), and, it is expected that *idptr will be set to a pointer to a driver internal structure which holds all the per-open state information, including the device pointer v. Return zero for success or a negative error code. The return value is returned to the application. The driver should do a MOD_INC_USE_COUNT on a successful open.

It is highly recommended to support no-I/O opens where it is possible and it makes sense. To support this you have to support multiple opens by filling in *idptr with a unique pointer for each open, and look for the O_NOIO flag in the flags parameter to open(). If this flag is set the this open should be marked as no-I/O. The remaining issues with no-I/O opens are related to correctly failing any operation that is not on the approved list in the API spec. Illegal ioctl calls and read() should return -EPERM; mmap() should return -ENODEV; poll() should return POLLERR.

Method: void close(void *id)

Called when the file descriptor is being closed. Passed the ID value assigned in the corresponding open. Stop the device. There is no return value. Videodev always returns 0 to the application. The driver should do MOD_DEC_USE_COUNT.

Method: long read(void *id, char *buf, unsigned long count, int noblock)

Called when the application calls read(). Passed the ID value assigned in the corresponding open, the buffer, the number of bytes of data requested, and whether the read should block or not if there is no data ready. The return value is the number of bytes read or a negative error code. The return value is returned to the application.

Method: long write(void *id, const char *buf, unsigned long count, int noblock)

Called when the application calls write(). Passed the ID value assigned in the corresponding open, the buffer, the number of bytes requested to be written, and whether write should block if the device is not ready to accept data. The return value is the number of bytes written or a negative error code. The return value is returned to the application.

Method: int ioctl(void *id, unsigned int cmd, void *arg)

Called when the application calls ioctl(). Passed the ID value assigned in the corresponding open, the ioctl code, and the argument to the command. Return zero for success or a negative error code. The return value is returned to the application. Drivers should return -EINVAL for ioctl codes in the API which the driver does not support. Drivers should return -ENOIOCTLCMD for all unknown ioctl codes.

Note that the arg parameter will point to kernel memory, and is not the orginal user-space pointer passed in from the application. Videodev will handle the copy_from_user() and copy_to_user() work as needed. Exceptions are if the argument buffer is larger than V4L2_MAX_IOCTL_SIZE (256 bytes), and the clips parameter of a VIDIOC_S_WIN ioctl.

It is important for drivers to handle VIDIOC_QUERYCTRL and to return -EINVAL if the id field is out of range. If your driver does not have any private controls, you need to return -EINVAL if the id field is V4L2_CID_PRIVATE_BASE. It is not necessary to fill in the v4l2_queryctrl category or group fields for pre-defined control IDs; videodev will do it. Videodev fills them in before the driver is called, so the driver can change the default values if desired.

For backward compatibility with applications written for the original API, videodev will translate the old ioctl codes and parameters into V4L2 ioctl calls. But videodev will first pass the ioctl to the driver. If the driver returns -ENOIOCTLCMD, then videodev will attempt to translate it to equivalent new ioctl calls. Therefore it is possible for a driver to handle old ioctl commands if desired for any reason.

Method: int mmap(void *id, struct vm_area_struct *vma)

Called when the application calls mmap(). Passed the ID value assigned in the corresponding open, and the virtual memory area structure. The V4L2 specification defines a general-purpose protocol for using mmap() for buffer mapping that all V4L2 devices need to follow. Return 0 for success or a negative error code. The return value is returned to the application. Return -EINVAL for an invalid parameter, or -ENODEV if the buffer could not be allocated.

The V4L2 memory mapping protocol allows for many buffers to be mapped individually, with the buffers organized into independent buffer sets. In the v4l2_buffer structure, the type field identifies the set the buffer belongs to and the index field identifies the individual buffer. It is not possible to pass a v4l2_buffer structure directly to the mmap method, so we use the vma->vm_offset field to identify the buffer to be mapped. You will fill in the offset field of video_buffer with a number that is unique for each buffer that can be mapped. The application will pass that number back to you in the vma->vm_offset field when it calls mmap(). Your mmap handler will look at vm_offset to determine which buffer is to be mapped. You need to invent a numbering scheme for your buffers for this purpose. Warning: the offset numbers should not be based on the length of the buffers or else the kernel might merge vma's leading to trouble. It is recommended that buffers located in system RAM be allocated in mmap. Buffers in system memory should be zero-filled when they are allocated so applications cannot gain access to any data that may be remaining in that memory.

Since V4L2 drivers are modules, you should implement the open and close vma->vm_ops method calls, and use MOD_INC_USE_COUNT in open, and MOD_DEC_USE_COUNT in close. The system does not automatically call the open method, so you need to call it from mmap. You should free a system memory buffer in close. Remember to set the V4L2_BUF_FLAG_MAPPED flag when a buffer is mapped, and clear it when it is unmapped. It is not possible to fail a munmap(), so if it is called while streaming is active, you should safely stop the hardware and dequeue the buffer before freeing it.

The videodev mmap wrapper will set up vma->vm_file before calling the V4L2 driver. Videodev will also increment the file usage count after the V4L2 driver's mmap method returns if the mmap is successful. The driver should not increment the file usage count.

The dimensions of a buffer are typically dependent on a data format setting. For example, video capture buffers are dependent on the capture image format. Typically, you would fail an attempt to change the format while a buffer is mapped. An exception is that for v4l backward compatibility, a VIDIOC_S_FMT call is permitted while video capture buffers are mapped, although such a call could not set a format that was larger than would fit in the buffers. The format can be changed after all buffers dependent on the format have been unmapped. After the format has been changed, it is required for the application to call VIDIOC_REQBUFS again, therefore you must make changing the format also undo the effect of VIDIOC_REQBUFS.

If you want to let applications map additional types of memory buffers or memory areas, or have multiple streams of data, this interface is trivially extensible: just add more values for the buffer type field. Values starting from V4L2_BUF_TYPE_PRIVATE_BASE are reserved for driver-private buffer types.

Method: int poll(void *id, struct file *file, poll_table *table)

Called when the application calls select(). Passed the ID value assigned in the corresponding open, the struct file pointer and the poll_table pointer. The latter two are needed for the poll_wait() system call. Return the regular POLL* flags according to the status. Return POLLERR for a no-I/O open.
 

V4L2 Video Capture Drivers

Non-capturing opens. It is highly recommended to support non-capturing opens on capture devices. This way we can have universal video control panel applications that can run alongside a capturing program. VIDIOC_S_PARM can't change the capturemode or timeperframe.

Video for Linux Backward Compatibility. The original API used mmap differently from V4L2. In the original API, a single mmap call mapped all the buffers contiguously in a single, long virtual memory area. It's not possible for the backward compatibility code in videodev to make a V4L2 driver function like a v4l driver in this regard, so if you want your driver to work well with applications that use the v4l API mmap you need to support mapping all the requested buffers in one mmap call. It's only used for video capture, V4L2_BUF_TYPE_CAPTURE, buffers. The compatibility layer will set the flag V4L2_BUF_REQ_CONTIG in the v4l2_requestbuffers type field in the VIDIOC_REQBUFS ioctl. If you support this contiguous mapping of buffers then make sure the flag is set in the type field when VIDIOC_REQBUFS returns. The number of buffers requested this way will most likey be two. Note: the CONTIG flag should not be set in v4l2_buffer structures, or anywhere else-- only v4l2_requestbuffers. On the next mmap call, allocate a block of memory big enough for all the buffers. The vma_close call will free and unmap all the buffers at once. The vm_offset field will be zero when contiguous mapping is used. The v4l2cap.c sample capture driver illustrates all of this.

If you don't support the V4L2_BUF_REQ_CONTIG flag, a v4l application will only be able to map one capture buffer, which may not be enough to get the capture performance the user expects.

V4L2 Kernel-Mode API

Driver Registration and Device Structures

int v4l2_register_device(struct v4l2_device *v)

Register the device structure pointer, and assign a minor device number.

Unregister the specified device.

Returns the V4L2_MAJOR_VERSION and V4L2_MINOR_VERSION that the videodev.o module was compiled with.

Returns the V4L2 device structure pointer corresponding to a minor device number.

Returns the V4L2 device structure pointer corresponding to a Linux file structure. Useful in vma->vm_ops handlers.

Returns the open ID pointer corresponding to a Linux file structure.

unsigned long v4l2_vmalloc_to_bus(void *virt)

Returns the bus address corresponding to the virtual address of memory that was allocated with vmalloc(). Useful for using vmalloc()ed memory as a DMA buffer.

Returns the page corresponding to a pointer into a buffer allocated with vmalloc(). Useful in the vma->vm_ops->nopage method when memory mapping a buffer allocated with vmalloc(). The return value from this function is suitable for use as the return value from vma->vm_ops->nopage.

Queues consist of a root node of type struct v4l2_queue, and element nodes of type struct v4l2_q_node. Allocate a struct v4l2_queue, for example in the device structure, to be the 'root' node. Pass a pointer to this root node to the queue functions when adding/deleting/etc. to the queue. The queue operations are protected by IRQ-safe, read-write spin locks, so they will work correctly when called from interrupt or in a multiprocessor environment. It is a good idea to call v4l2_q_init() before using a queue to put it in a safe initial state.

void v4l2_q_init(struct v4l2_queue *q)

Initialize the queue to the empty state.

Add a node to the head of the queue.

Add a node to the tail of the queue.

Delete a node from the head of the queue and return a pointer to the node. Returns NULL if queue is empty.

Delete a node from the tail of the queue and return a pointer to the node. Returns NULL if queue is empty.

Return a pointer to the node at the head of the queue without changing the queue. Returns NULL if queue is empty.

Return a pointer to the node at the tail of the queue without changing the queue. Returns NULL if queue is empty.

Scan the queue for the node specified and remove it from the queue if it is there. Returns the node or NULL if it was not there.

Returns 1 if there is exactly one item in the queue, 0 if there is more than one item, or -1 otherwise.

These functions are for getting and using time stamps on buffers. Timestamps have the data type stamp_t , which is a signed 64-bit integer. Timestamps use units of nanoseconds. (2⁶³ nanoseconds is over 292 years.) I hope to add a clock to the kernel like SGI's Unadjusted System Time, but in the mean time the time-of-day clock will be used as the time base.

Drivers should always call v4l2_masterclock_gettime() to get the current time.

The purpose of v4l2_masterclock_register() is so that it is possible to substitute a new time base function. This could be used if you had another time base such as a clock on an add-in card that you wanted to use.

int v4l2_masterclock_register (struct v4l2_clock *clock)

Register a master clock function. Only one master clock can be registered at a time for the whole system. Returns zero if the clock was registered, -1 if it was not (another clock is already registered).

Unregister a master clock function.

Get the current time according to the master clock.

unsigned long v4l2_timestamp_divide (stamp_t t, unsigned long p_100ns)

Divide the time interval expressed in t by the time interval expressed by p_100ns (which is in 100ns units), end return the quotient, rounded to the nearest whole number. WARNING: On x86 systems, this will OOPS if the quotient is greater than 2³².

"Correct" the time interval expressed in t by changing it to the nearest multiple of the time interval expressed by p_100ns (which is in 100ns units). Returns the multiple.

Operations on Video Standard Structures

These are the recommended functions for constructing and parsing v4l2_standard structures. If you are only supporting established standard formats then these functions can do almost all the work, and you will probably never need to touch any v4l2_standard fields in your code besides id, or maybe transmission if you deal with RF signals.

unsigned int v4l2_video_std_fps (struct v4l2_standard *vs)

Convert the framerate.numerator and framerate.denominator fields to a frames-per-second value, rounded to the nearest whole number.

Convert the framerate.numerator and framerate.denominator fields to a time per frame value expressed in 100ns units.

Examine a v4l2_standard structure, verify the validity of the numeric fields, and fill in the name field if the format is one of the recognized standards. Returns a negative number if the format is certainly invalid, returns the V4L2_STD_* value for recognized standards, and returns zero for all remaining cases. You may have to do some further tests if you are supporting non-standard formats in your driver.

Takes the id and transmission values and fills in all the fields of the v4l2_standard structure. Returns the id value back on success, or a negative error code.

u32 v4l2_math_div6432 (u64 a, u32 d, u32 *r)

Divide a 64-bit number by a 32-bit number, return the quotient, and return the remainder in r if r is not NULL. If d == 0 a divide fault occurs. If the quotient >= 2³² a divide fault may occur depending on the processor. [This requires __asm__ code on the i386, and may also for other 32-bit architectures. It should probably work on Alpha or any native 64-bit machine. I have only written i386 code.]