HOWTO easily use NASM into windows C/C++ applications

After installed NASM and configured our VisualStudio to use it, we are now able to use it in our C/C++ projects.

Assembler usually has a huge advantage in terms of performance at the cost of rigid code usage.
You cannot use it in both 32-bits and 64-bits versions of the same C/C++ code.  Each version shall fulfill several different ‘environmental’ requirements that cannot be ignored at all.

Even with all these complications, the benefits can be outstanding for a processing-hungry task, justifying all the efforts needed.
For example, I re-wrote my AES encryption library using SSE3 + AESNI instructions now going 200-300 times faster.

Changes from C/C++ side

Support we choose to re-write the following function in Assembler:

int MyFunction (const char *pszText, int val)
{
    return (int) *pszText + val;
}

First of all, we have to declare it as extern function.

extern MyFunction (const char *pszText, int val);

In C++ programs, we have also to exclude it from the name-mangling process, by:

extern "C" int MyFunction (const char *pszText, int val);

This is the only needed step to use it in our C/C++ program.

Writing down our assembler function

++ 32 bits version

Add a new .asm file to your project (for example: ‘nasm32_MyFunction.asm’).

Our first line is:

bits 32

It is needed to define the ‘geometry’ of our assembler piece of code.

If we need to define and use some ‘static’ data, we have to define a data section

section .data

and include them here.

Then we define the code section

section .text

then we have to define our function name as a global symbol to use it from ‘outside’ this module.

global _MyFunction

and write down our function ‘definition’:

_MyFunction:
%push proc_ctx
%stacksize flat
%arg pszText:DWORD, len:DWORD

then we have to create a function ‘context’ in the stack by using:

push ebp
mov ebp,esp

or

enter 0,0

then we have to preserve the values of the used reserved registers (EBX, ESI, EDI)

push ebx

now we can add the function processing code

mov ebx,[ebp + pszText]
xor eax,eax
mov eal,[ebx]
add eax,[ebp + val]

the we have to restore used reserved registers

pop ebx

and cleanup the function context and return

leave
ret

At the end our .asm file should be as below:

bits 32

section .text

; int MyFunction (const char *pszText, int val)
global _MyFunction
_MyFunction:
%push proc_ctx
    %stacksize flat
    %arg pszText:DWORD, len:DWORD

    push ebp
    mov ebp,esp
    ; save reserved registers
    push ebx
    ; process data
 xor eax,eax
    mov ebx,[ebp + pszText]
    xor eax,eax
    mov eal,[ebx]
    add eax,[ebp + val]
    ; restore reserved registers
    pop ebx
    ; cleanup context
    leave
    ret
%pop

++ the 64 bits version

The 64-bits version has different conventions for the parameters, … (see this MS article).

Mainly the parameters are not stored in the stack anymore.

bits 64

section .text

; int MyFunction (const char *pszText, int val)
global _MyFunction
; pszText = RCX
; val = RDX
_MyFunction:
%push proc_ctx

    ; process data
    xor rax,rax
    mov al,[rcx]
    add rax,rdx
    ret
%pop

As you can see, the 64-bits version of our function doesn’t use anything about context and so on allowing us to reduce our implementation to the bare-bone.

We cannot avoid it in case we used ‘local variables’ or our function is passing many parameters.

Defining and accessing to ‘local variables’

Local variables may be useful especially in 32-bits assembler procedures when CPU registers are quite limited.

The common solution is to reserve some space into the stack frame for such variables.

; int MyFunction (const char *pszText, int val)
global _MyFunction
_MyFunction:
%push proc_ctx
    %stacksize flat
    %arg pszText:DWORD, len:DWORD
    %assign %$localsize 0              ; define this before defining the local vars
    %local var_b:DWORD                 ; defined local variables
    %local var_a:DWORD

    enter %$localsize, 0
    ...
    mov eax,[ebp + var_a]              ; load 'var_a' value
    ...
    leave
    ret
%pop

Finally, my advice is to read the NASM documentation to take advantage of all the included features.