[MultiMedia] How to Build the Filter Graph

Building the Filter Graph

1. Filter Graph Manager
• CLSID_FilterGraph: Creates the Filter Graph Manager on a shared worker thread. (通常都用CLSID_FilterGraph)
• CLSID_FilterGraphNoThread: Creates the Filter Graph Manager on the application thread. 
2. IGraphBuilder Interface
• Connect: Connects two pins. If they will not connect directly, this method connects them with intervening transforms.
• Render: Adds a chain of filters to a specified output pin to render it. (將Source的output pin 試著connect到已Add的filter)
• RenderFile: Builds a filter graph that renders the specified file. (也就是AutoRender)
• AddSourceFilter: Adds a source filter to the filter graph for a specific file. 
3. IMediaControl Interface
• Run: Runs all the filters in the filter graph.
• Pause: Pauses all filters in the filter graph.
• Stop: Stops all the filters in the filter graph.
• StopWhenReady: Pauses the filter graph, allowing filters to queue data, and then stops the filter graph.
• GetState: Retrieves the state of the filter graph. 
4. IMediaEventEx Interface
• SetNotifyWindow: Registers a window to process event notifications.
• SetNotifyFlags: Enables or disables event notifications.
• GetNotifyFlags: Determines whether event notifications are enabled. 

Reference

• Filter Graph Manager 
• Graph-Building Components

[MultiMedia] What's New for DirectShow in Windows 7 - Filters

Microsoft MPEG-1/DD/AAC Audio Decoder


This filter decodes the following audio formats:

• MPEG-1 audio layers I and II.
• Backward-compatible MPEG-2 audio, layers I and II (ISO/IEC 13818-3), mono and stereo only.
• Advanced Audio Coding (AAC) Low Complexity (LC) profile.
• High-Efficiency AAC (HE-AAC) version 1 and version 2.
• Pass-through Digital Theater Systems (DTS) for digital output.
• LPCM, mono and stereo only, with or without PES headers.
• Dolby Digital.
• Dolby Digital Plus, including conversion from Dolby Digital Plus to Dolby Digital for digital output.

Note This filter is not supported on IA-64-based platforms.

In the registry, the friendly name of this filter is "Microsoft DTV-DVD Audio Decoder".

Microsoft MPEG-2 Video Decoder

• This filter decodes MPEG-1, MPEG-2, H.264 video.

• Note Decoding of H.264 video requires Windows 7.

Note This filter is not supported on IA-64-based platforms.

In the registry, the friendly name of this filter is "Microsoft DTV-DVD Video Decoder".

[MultiMedia] DirectShow Note

This is a studying note for DirectShow. It summarizes some key knowledge of DirectShow and a studying roadmap. Current progress is Building the Filter Graph (2010/03/08).

DirectShow

• The Microsoft DirectShow application programming interface (API) is a media-streaming architecture for the Microsoft Windows platform.
• DirectShow simplifies media playback, format conversion, and capture tasks.
• DirectShow is based on the Component Object Model (COM).
• DXVA: DirectX Video Acceleration.
• EVR: Enhanced Video Renderer.
• VMR: Video Mixing Renderer.
• DMOs: Microsoft DirectX Media Objects.

Filters and Filter Graphs

The building block of DirectShow is a software component called a filter. A filter is a software component that performs some operation on a multimedia stream. For example, DirectShow filters can

• read files
• get video from a video capture device
• decode various stream formats, such as MPEG-1 video
• pass data to the graphics or sound card

Filters receive input and produce output.

In DirectShow, an application performs any task by connecting chains of filters together, so that the output from one filter becomes the input for another. A set of connected filters is called a filter graph.

How To Play a File

See example and sample code on MSDN.

The DirectShow Solution

See DirectShow component relationship diagram on MSDN.

About DirectShow Filters

The connection points are also COM objects, called pins. Filters use pins to move data from one filter the next. In DirectShow, a set of filters is called a filter graph.

Filters have three possible states: running, stopped, and paused. When a filter is running, it processes media data. When it is stopped, it stops processing data. The paused state is used to cue data before running.

The primary function of most filters is to process and deliver media data. How that occurs depends on the type of filter:

• A push source has a worker thread that continuously fills samples with data and delivers them downstream.
• A pull source waits for its downstream neighbor to request a sample. It responds by writing data into a sample and delivering the sample to the downstream filter. The downstream filter creates the thread that drives the data flow.
• A transform filter has samples delivered to it by its upstream neighbor. When it receives a sample, it processes the data and delivers it downstream.
• A renderer filter receives samples from upstream, and schedules them for rendering based on the time stamps.

About the Filter Graph Manager

The Filter Graph Manager is a COM object that controls the filters in a filter graph. It performs many functions, including the following:

• State changes: Coordinating state changes among the filters.
• Reference clock: Establishing a reference clock.
• Graph events: Communicating events back to the application.
• Graph-building methods: Providing methods for applications to build the filter graph.

About Media Types
 

The media type is a universal and extensible way to describe digital media formats. When two filters connect, they agree on a media type. The media type identifies what kind of data the upstream filter will deliver to the downstream filter, and the physical layout of the data. If two filters cannot agree on a media type, they will not connect.

 
Media types are defined using the AM_MEDIA_TYPE structure. This structure contains the following information:

• 
  
  Major type: The major type is a GUID that defines the overall category of the data. Major types include video, audio, unparsed byte stream, MIDI data, and so forth.
• 
  
  Subtype: The subtype is another GUID, which further defines the format. For example, within the video major type, there are subtypes for RGB-24, RGB-32, UYVY, and so forth. Within audio, there is PCM audio, MPEG-1 payload, and others. The subtype provides more information than the major type, but it does not define everything about the format. For example, video subtypes do not define the image size or the frame rate. These are defined by the format block, described below.
• 
  
  Format block: The format block is a block of data that describes the format in detail. The format block is allocated separately from the AM_MEDIA_TYPE structure. The pbFormat member of the AM_MEDIA_TYPE structure points to the format block. The pbFormat member is typed void* because the layout of the format block changes depending on the media type. For example, PCM audio uses a WAVEFORMATEX structure. Video uses various structures, including VIDEOINFOHEADER and VIDEOINFOHEADER2. The formattype member of the AM_MEDIA_TYPE structure is a GUID that specifies which structure is contained in the format block. Each format structure is assigned a GUID. The cbFormat member specifies the size of the format block. Always check these values before dereferencing the pbFormat pointer.

About Media Samples and Allocators

Filters deliver data across pin connections. Data moves from the output pin of one filter to the input pin of another filter. The most common way for the output pin to deliver the data is by calling the IMemInputPin::Receive method on the input, although a few other mechanisms exist as well. More detail see MSDN.

How Hardware Devices Participate in the Filter Graph

Wrapper Filters

All DirectShow filters are user mode software components. In order for a kernel mode hardware device, such as a video capture card, to join a DirectShow filter graph, the device must be represented as a user-mode filter. This function is performed by specialized "wrapper" filters provided with DirectShow. DirectShow also provides a filter called KsProxy, which can represent any type of Windows Driver Model (WDM) streaming device. Hardware vendors can extend KsProxy to support custom functionality, by providing a Ksproxy plug-in, which is a COM object aggregated by KsProxy.

The wrapper filters expose COM interfaces that represent the capabilities of the device. The application uses these interfaces to pass information to and from the filter. The filter translates the COM method calls into device driver calls, passes that information to the driver in kernel mode, and then translates the result back to the application. Some filters support custom driver properties through the IKsPropertySet interface.

For application developers, wrapper filters enable the application to control devices just as they control any other DirectShow filter. No special programming is required; the details of communicating with the kernel-mode device are encapsulated within the filter.

References

• MSDN: DirectShow 

[MultiMedia] Mpeg Format

Mpeg Format

• Mpeg TS: Transport Stream. 
• Mpeg PS: Program Stream.

Mpeg TS Format

• MPEG_TS_HD_NA: 192 bytes with 4 bytes zero time code.
• MPEG_TS_HD_NA_ISO: 188 bytes without time code.
• MPEG_TS_HD_NA_T: 192 bytes with 4 bytes valid time code.
• MPEG_TS_JP_T: 192 bytes with 4 bytes valid time code. Has a CP Header.

MPEG_TS_JP_T CP Header Structure

• 14 bytes header begin with 0x00 (byte_length is 10~13 bytes)
• 4 bytes time code
• 188 bytes payload begin with 0x47

MPEG_TSJP_T CP Header Structure Example

• First CP Header begin at 00 00 00 00: 00 00 00 00 00 00 00 00 00 00 [00 0F FF C0] => Byte Length = 00 0F FF C0 means the length of the first segment (Time Code (4) + Payload (188)) is 00 0F FF C0
• Time Code (4 bytes): BE 84 0A DC
• Payload (188 bytes): 47 40 00 11 00 00 B0 11 40 F2 C1 00 00 00 00 E0 ...
• Second CP Header begin at 00 0F FF CD: First CP Header Size + Byte Length = 14 + 00 0F FF C0 = 00 0F FF CD

References

• MPEG transport stream in wiki
• MPEG program stream in wiki

[C++] Windows Registry Key

Example

int nRet;
HKEY hKey;
DWORD dwSize = 256;  // Note 1
char pszKeyName[256] = {0};

nRet = RegOpenKeyEx(HKEY_LOCAL_MACHINE,"Software\\PathA",0,KEY_READ,&hKey);
if (nRet == ERROR_SUCCESS)
    nRet = RegQueryValueEx(hKey,"KeyName",NULL,NULL,(BYTE *)g_pszKeyName,&dwSize);
nRet = RegCloseKey(hKey); // Note 2

dwSize = 256; // Note 3

nRet = RegOpenKeyEx(HKEY_LOCAL_MACHINE,"Software\\PathB",0,KEY_READ,&hKey);
if(nRet == ERROR_SUCCESS)
    nRet = RegQueryValueEx(hKey,"KeyName",NULL,NULL,(BYTE *)g_pszKeyName,&dwSize);
nRet = RegCloseKey(hKey);

Note

• Note 1: dwSize 和 pszKeyName[]要夠大.
• Note 2: RegOpenKeyEx後要記得RegCloseKey, 不然RegOpenKeyEx 65535次會無法RegOpenKeyEx.
• Note 3: 重複使用dwSize 要reset size不然dwSize會為之前RegQueryValueEx的值.

References

• RegOpenKeyEx
• RegQueryValueEx
• RegCloseKey

[Java] javac

問題

• 'javac' 不是內部或外部命令、可執行的程式或批次檔。

解法

• 在「我的電腦」按右鍵 <「內容」<「進階」<「環境變數」
• 在系統變數那邊找到path，按「編輯」
• 加上C:\Program Files\Java\jdk1.5.0_12\bin\

[Java] Performance Tuning

Performance Tuning:

• Know How:
  • Designing for Performance
    • Avoid Creating Objects.
      • Generally speaking, avoid creating short-term temporary objects if you can. Fewer objects created mean less-frequent garbage collection, which has a direct impact on user experience.
    • Performance Myths
      • It was cheaper to invoke methods on a HashMap map than a Map map,
    • Prefer Static Over Virtual
      • If you don't need to access an object's fields, make your method static. Invocations will be about 15%-20% faster. It's also good practice, because you can tell from the method signature that calling the method can't alter the object's state.
    • Use Static Final For Constants
      • We can improve matters with the "final" keyword: static final String strVal = "Hello, world!"; instead of static String strVal = "Hello, world!";
      • strVal will use a relatively inexpensive "string constant" instruction instead of a field lookup.
    • Use Enhanced For Loop Syntax
      • Algorithm Performance C > B > A. Ex: A. 在for loop用length. B. 不在for loop用length, 用local variable存length. C. 用Foo a : mArray.
    • Avoid Enums Where You Only Need Ints
    • Use Package Scope with Inner Classes
    • Use Floating-Point Judiciously
    • Know And Use The Libraries
      • Ex: String.indexOf
    • Use Native Methods Judiciously
  • Use StringBuilder to improve string editing performance.
  • 不再使用的物件要盡早設定為 null，以便早點被當成垃圾清掉。
  • Use global variable to reuse memory.
• Reference:
  • http://developer.android.com/guide/practices/design/performance.html
  • http://www.oreilly.com.tw/column_sleepless.php?id=j005
  • http://blog.javia.org/android-allocation-free-code-avoids-the-gc-freeze/
  • http://j2se.myweb.hinet.net/article/java/010.htm

[Java] While loop, For loop and Iterator Performance Test

http://www.mkyong.com/java/while-loop-for-loop-and-iterator-performance-test-java/

結論是While loop, For loop 比 Iterator Performance好很多