Debugging tricks (Visual Studio)

1. Display TIB(Thread Information Block) in watch:
   - (_TEB*)$tib

Profilers for Windows

1. AMD CodeAnalyst
2. Visual Studio Team System 
3. Very Sleepy
4. micro-profiler
5. Xperf 

Consistent Overhead Byte Stuffing (@Wiki)

Consistent Overhead Byte Stuffing

From Wikipedia, the free encyclopedia

Consistent Overhead Byte Stuffing (COBS) is an algorithm for encoding data bytes that results in efficient, reliable, unambiguous packet framing regardless of packet content, thus making it easy for receiving applications to recover from malformed packets.
Byte stuffing is a process that transforms a sequence of data bytes that may contain 'illegal' or 'reserved' values into a potentially longer sequence that contains no occurrences of those values. The extra length of the transformed sequence is typically referred to as the overhead of the algorithm. The COBS algorithm tightly bounds the worst case overhead, limiting it to no more than one byte in 254. The algorithm is computationally inexpensive and its average overhead is low compared to other unambiguous framing algorithms.^[1]

Contents 1 Packet framing and stuffing 1.1 Zero Pair Elimination 2 Packet format 3 Implementation 4 References 5 External Links

Packet framing and stuffing

When packet data is sent over any serial medium, a protocol is needed by which to demarcate packet boundaries. This is done by using a special bit-sequence or character value to indicate where the boundaries between packets fall. Data stuffing is the process that transforms the packet data before transmission to eliminate any accidental occurrences of that special framing marker, so that when the receiver detects the marker, it knows, without any ambiguity, that it does indeed indicate a boundary between packets.
COBS takes an input consisting of bytes in the range [0,255] and produces an output consisting of bytes only in the range [1,255]. Having eliminated all zero bytes from the data, a zero byte can now be used unambiguously to mark boundaries between packets. This allows the receiver to synchronize reliably with the beginning of the next packet, even after an error. It also allows new listeners, which might join a broadcast stream at any time, to reliably detect the beginning of the first complete packet in the received byte stream.
With COBS, all packets up to 254 bytes in length are encoded with an overhead of exactly one byte. For packets over 254 bytes in length the overhead is at most one byte for every 254 bytes of packet data. The maximum overhead is therefore roughly 0.4% of the packet size, rounded up to a whole number of bytes. COBS encoding has low overhead (on average 0.23% of the packet size, rounded up to a whole number of bytes) and furthermore, for packets of any given length, the amount of overhead is virtually constant, regardless of the packet contents.

Zero Pair Elimination

An optimization that can reduce overhead for common payloads which contain pairs of zero bytes is to reduce the maximum encodable sequence length, freeing some codes to encode sequences terminated by pairs of zeros. In this case, bytes in the range [1,223] have the same meaning in as in the normal mode, the code 224 is used to encode a sequence of 223 bytes with no zero termination, and the remaining codes [225,255] encode sequences of length [1,30] terminated by a pair of zero bytes. This variation can achieve negative overhead (compression) for some sequences however it does complicate the en/decoding process.

Packet format

COBS encodes the input data as a series of variable length blocks. Each block, which may contain from 1 to 255 bytes, begins with a single byte that specifies the number of bytes in the block (including the length byte).
When decoding, a zero byte is appended to the decoded output after each block. As a special case, no zero is added after a block which begins with 0xFF.
Example encodings (block contents marked up in bold):

	Plaintext	Encoded with COBS
1.	0x00	0x01 0x01
2.	0x11 0x22 0x00 0x33	0x03 0x11 0x22 0x02 0x33
3.	0x11 0x00 0x00 0x00	0x02 0x11 0x01 0x01 0x01
4.	0x01 0x02 ... 0xFF	0xFF 0x01 0x02 ... 0xFE 0x02 0xFF

There exists one complication in this format: in the case 2. above, an extra 0x00 appears at the end of the decoded output. The only way to encode a block that does not end in zero is for it to have 254 bytes of contents, but the last block may be shorter than that. To solve this issue, a single trailing zero, if present, is removed by the decoder. If the real plaintext ends in a zero, an additional zero is added after it.

Implementation

/*
 * StuffData byte stuffs "length" bytes of
 * data at the location pointed to by "ptr",
 * writing the output to the location pointed
 * to by "dst".
 */
 
#define FinishBlock(X) (*code_ptr = (X), code_ptr = dst++, code = 0x01)
 
void StuffData(const unsigned char *ptr,
unsigned long length, unsigned char *dst)
{
  const unsigned char *end = ptr + length;
  unsigned char *code_ptr = dst++;
  unsigned char code = 0x01;
 
  while (ptr < end)
  {
    if (*ptr == 0)
      FinishBlock(code);
    else
    {
      *dst++ = *ptr;
      code++;
      if (code == 0xFF)
        FinishBlock(code);
    }
    ptr++;
  }
 
  FinishBlock(code);
}
 
/*
 * UnStuffData decodes "length" bytes of
 * data at the location pointed to by "ptr",
 * writing the output to the location pointed
 * to by "dst".
 */
 
void UnStuffData(const unsigned char *ptr,
unsigned long length, unsigned char *dst)
{
  const unsigned char *end = ptr + length;
  while (ptr < end)
  {
    int i, code = *ptr++;
    for (i=1; i<code; i++)
      *dst++ = *ptr++;
    if (code < 0xFF)
      *dst++ = 0;
  }
}

References

^ Cheshire, Stuart; Baker, Mary. "Consistent Overhead Byte Stuffing". ACM. Retrieved November 23, 2010.

Booting LINUX form Live USB

1. Use YUMI to prepare USB stick (http://www.pendrivelinux.com/yumi-multiboot-usb-creator/)
2. Assure you are using FAT (not FAT32) file system. For some motherboards FAT32 doesn't work. (on windows: format A: /fs:FAT)
3. Some kernels require boot option: mem=4gb

Collecting User-Mode Dumps

http://msdn.microsoft.com/en-us/library/windows/desktop/bb787181%28v=vs.85%29.aspx
http://blogs.microsoft.co.il/blogs/sasha/archive/2009/10/19/configuring-automatic-crash-dumps.aspx

Starting with Windows Server 2008 and Windows Vista with Service Pack 1 (SP1), Windows Error Reporting (WER) can be configured so that full user-mode dumps are collected and stored locally after a user-mode application crashes. Applications that do their own custom crash reporting, including .NET applications, are not supported by this feature.
This feature is not enabled by default. Enabling the feature requires administrator privileges. To enable and configure the feature, use the following registry values under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps key.

Value	Description	Type	Default value
DumpFolder	The path where the dump files are to be stored. If you do not use the default path, then make sure that the folder contains ACLs that allow the crashing process to write data to the folder. For service crashes, the dump is written to service specific profile folders depending on the service account used. For example, the profile folder for System services is %WINDIR%\System32\Config\SystemProfile. For Network and Local Services, the folder is %WINDIR%\ServiceProfiles.	REG_EXPAND_SZ	%LOCALAPPDATA%\CrashDumps
DumpCount	The maximum number of dump files in the folder. When the maximum value is exceeded, the oldest dump file in the folder will be replaced with the new dump file.	REG_DWORD	10
DumpType	Specify one of the following dump types: 0: Custom dump 1: Mini dump 2: Full dump	REG_DWORD	1
CustomDumpFlags	The custom dump options to be used. This value is used only when DumpType is set to 0. The options are a bitwise combination of the MINIDUMP_TYPE enumeration values.	REG_DWORD	MiniDumpWithDataSegs | MiniDumpWithUnloadedModules | MiniDumpWithProcessThreadData.

These registry values represent the global settings. You can also provide per-application settings that override the global settings. To create a per-application setting, create a new key for your application under HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\LocalDumps (for example, HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\LocalDumps\MyApplication.exe). Add your dump settings under the MyApplication.exe key. If your application crashes, WER will first read the global settings, and then will override any of the settings with your application-specific settings.
After an application crashes and prior to its termination, the system will check the registry settings to determine whether a local dump is to be collected. After the dump collection has completed, the application will be allowed to terminate normally. If the application supports recovery, the local dump is collected before the recovery callback is called.
These dumps are configured and controlled independently of the rest of the WER infrastructure. You can make use of the local dump collection even if WER is disabled or if the user cancels WER reporting. The local dump can be different than the dump sent to Microsoft.

To have it working on my Windows 7 x64 I had to set auto="0" in:

HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows NT\CurrentVersion\AeDebug

Performance of sorting procedures

1. Results

   Those results were generated with vs2012 Release configuration and full optimization: (\Ox):
   sort (tmp.begin(),tmp.end(),less<int>()): 399.258 milliseconds
   sort (tmp.begin(),tmp.end()): 402.652 milliseconds
   sort (tmp.begin(),tmp.end(), (lessFunction<int>)): 2.11581 seconds
   qsort (&tmp[0],tmp.size(),sizeof(int), compare): 2.43363 seconds
   

2. Code

   #include <windows.h>
   
   #include <vector>
   #include <cstdlib>
   #include <iostream>
   #include <ostream>
   #include <sstream>
   #include <set>
   #include <string>
   #include <algorithm>
   #include <numeric>
   #include <future>
   #include <functional>
   
   using namespace std;
   size_t MAX_SIZE = (1 << 28) + 1;
   
   long long timer() {
       LARGE_INTEGER li;
       QueryPerformanceCounter(&li);
       return li.QuadPart;
   }
   long long frequency() {
       LARGE_INTEGER li;
       QueryPerformanceFrequency(&li);
       return li.QuadPart;
   }
   string format_time(const long long dt) {
       ostringstream oss;
       if ((dt) * 1000 < frequency()) {
           oss << (dt) * 1000000.0 / frequency() << " microseconds";
       } else if (dt < frequency()) {
           oss << (dt) * 1000.0 / frequency() << " milliseconds";
       } else {
           oss << (dt) * 1.0 / frequency() << " seconds";
       }
       return oss.str();
   }
   
   void fillRandomVector (vector <int> *v)
   {
       srand (0);
       for (auto i:*v)
           i = rand();
   }
   
   template <typename T>
   bool lessFunction(const T &d1, const T &d2)
   {
       return d1 < d2;
   }
   
   int compare (const void * a, const void * b)
   {
       return ( *(int*)a - *(int*)b );
   }
   
   int main()
   {
       typedef pair<long long, string> tTiemrResult;
       vector <tTiemrResult> tx;
   
       vector <int> tmp (MAX_SIZE);
   
       fillRandomVector (&tmp);
       tx.push_back(tTiemrResult(timer(),"start"));
       sort (tmp.begin(),tmp.end());
       tx.push_back(tTiemrResult(timer(),"sort (tmp.begin(),tmp.end())"));
   
       fillRandomVector (&tmp);
       tx.push_back(tTiemrResult(timer(),"start"));
       sort (tmp.begin(),tmp.end(),less<int>());
       tx.push_back(tTiemrResult(timer(),"sort (tmp.begin(),tmp.end(),less<int>())"));
   
       fillRandomVector (&tmp);
       tx.push_back(tTiemrResult(timer(),"start"));
       sort (tmp.begin(),tmp.end(), (lessFunction<int>));
       tx.push_back(tTiemrResult(timer(),"sort (tmp.begin(),tmp.end(), (lessFunction<int>))"));
   
       fillRandomVector (&tmp);
       tx.push_back(tTiemrResult(timer(),"start"));
       qsort (&tmp[0],tmp.size(),sizeof(int), compare);
       tx.push_back(tTiemrResult(timer(),"qsort (&tmp[0],tmp.size(),sizeof(int), compare)"));
   
       vector <tTiemrResult> tx2;
       for (size_t i=1; i < tx.size() ; i++)
           tx2.push_back(tTiemrResult(tx[i].first - tx[i-1].first, tx[i].second));
       sort (tx2.begin(), tx2.end());
       for (auto i:tx2)
           cout<<i.second<<": "<< format_time (i.first)<<endl;
       return 0;
   }
   

Performance of summing procedures.

1. Results
   Those results were generated with vs2012 Release configuration and full optimization:
   
   /c /Zi /nologo /W4 /WX- /Ox /Oy- /GL /D WIN32 /D NDEBUG /D _CONSOLE /D _UNICODE /D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fo"RELEASE\\" /Fd"RELEASE\VC110.PDB" /Gd /TP /analyze-
   
   
   asyncTabIndexedSum: 1.69146 seconds (2 cores)
   tabIndexedSum: 2.37756 seconds (optimized for SSE)
   vectorIndexedSum: 3.04009 seconds
   accumulateTabSum: 3.24292 seconds
   accumulateVectorSum: 3.31494 seconds
   vectorRangedSum: 3.35942 seconds
   tabPointeredSum: 3.45808 seconds
2. Code
   
   #include <tchar.h>
   #include <windows.h>
   #include <vector>
   #include <cstdlib>
   #include <iostream>
   #include <ostream>
   #include <sstream>
   #include <set>
   #include <string>
   #include <algorithm>
   #include <numeric>
   #include <future>
   using namespace std;
   size_t MAX_SIZE = (1 << 28) + 1;
   size_t MAX_REPEATS = 10;
   long long timer() {
       LARGE_INTEGER li;
       QueryPerformanceCounter(&li);
       return li.QuadPart;
   }
   long long frequency() {
       LARGE_INTEGER li;
       QueryPerformanceFrequency(&li);
       return li.QuadPart;
   }
   string format_time(const long long dt) {
       ostringstream oss;
       if ((dt) * 1000 < frequency()) {
           oss << (dt) * 1000000.0 / frequency() << " microseconds";
       } else if (dt < frequency()) {
           oss << (dt) * 1000.0 / frequency() << " milliseconds";
       } else {
           oss << (dt) * 1.0 / frequency() << " seconds";
       }
       return oss.str();
   }
   int tabIndexedSum (const int tab[], size_t tabSize)
   {
       int sum = 0;
       for (size_t i = 0; i<tabSize; ++i)
           sum += tab[i];
       return sum;
   }
   int tabPointeredSum (const int *tabBegin, const int *tabEnd)
   {
       int sum = 0;
       for (const int *i = tabBegin; i < tabEnd; ++i)
           sum += *i;
       return sum;
   }
   int vectorIndexedSum (const vector <int> &tab)
   {
       int sum = 0;
       for (size_t i=0;i<tab.size();++i)
           sum += tab[i];
       return sum;
   }
   int vectorRangedSum (const vector <int> &tab)
   {
       int sum = 0;
       for (auto i:tab)
           sum += i;
       return sum;
   }
   int accumulateVectorSum (const vector <int> &tab)
   {
       return accumulate (tab.begin(), tab.end(), 0);
   }
   int accumulateTabSum (const int tab[], size_t tabSize)
   {
       return accumulate (tab, tab + tabSize, 0);
   }
   int asyncTabIndexedSum (const int *tab, size_t tabSize)
   {
       size_t halfSize = tabSize / 2;
       auto s1 = async(launch::async, tabIndexedSum, tab, halfSize);
       auto s2 = async(launch::async, tabIndexedSum, tab +  halfSize, tabSize - halfSize);
       return s1.get() + s2.get();
   }
   int _tmain(int argc, _TCHAR argv[])
   {
       typedef pair<long long, string> tTiemrResult;
       vector <tTiemrResult> tx;
       vector <int> results;
       vector <int> tmp (MAX_SIZE);
       for (size_t i = 0; i < MAX_SIZE; ++i)
           tmp [i] = rand();
     
       tx.push_back(tTiemrResult(timer(),"start"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(tabPointeredSum (&tmp[0], &tmp[0] + MAX_SIZE));
       tx.push_back(tTiemrResult(timer(),"tabPointeredSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(tabIndexedSum (&tmp[0], MAX_SIZE));
       tx.push_back(tTiemrResult(timer(),"tabIndexedSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(accumulateTabSum (&tmp[0], MAX_SIZE));
       tx.push_back(tTiemrResult(timer(),"accumulateTabSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(asyncTabIndexedSum (&tmp[0], MAX_SIZE));
       tx.push_back(tTiemrResult(timer(),"asyncTabIndexedSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(vectorIndexedSum (tmp));
       tx.push_back(tTiemrResult(timer(),"vectorIndexedSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(vectorRangedSum (tmp));
       tx.push_back(tTiemrResult(timer(),"vectorRangedSum"));
       for (size_t i=0;i<MAX_REPEATS;++i)
           results.push_back(accumulateVectorSum (tmp));
       tx.push_back(tTiemrResult(timer(),"accumulateVectorSum"));
       cout << "resultSet: ";
       for (auto i:set <int> (results.begin(),results.end()))
            cout<< i<<", ";
       cout << endl;
       vector <tTiemrResult> tx2;
       for (size_t i=1; i < tx.size() ; i++)
           tx2.push_back(tTiemrResult(tx[i].first - tx[i-1].first, tx[i].second));
       sort (tx2.begin(), tx2.end());
       for (auto i:tx2)
           cout<<i.second<<": "<< format_time (i.first)<<endl;
       return 0;
   }