Signal Processing is a subfield of electrical engineering that focuses on the analysis, manipulation, and interpretation of signals—functions that convey information about physical phenomena. Signals can be analog (continuous in time and amplitude) or digital (discrete in time and quantized in amplitude). The discipline encompasses mathematical and algorithmic techniques for filtering, transforming, compressing, and extracting features from signals, with applications ranging from audio and image processing to communications and biomedical systems.
1 Fundamentals
1.1 Definition and Scope of Signal Processing
Signal processing is the science of representing, transforming, and manipulating signals to extract useful information or to improve their quality. Its scope includes both theoretical foundations—such as linear system theory and statistical inference—and practical implementations in hardware or software. Applications span telecommunications, radar, audio engineering, medical imaging, and control systems.
1.2 Classification of Signals
Signals are classified according to their temporal nature, predictability, and periodicity.
1.2.1 Continuous-Time vs. Discrete-Time
A continuous-time signal is defined for every instant of time, typically represented by a function \(x(t)\). A discrete-time signal is defined only at discrete time instants, represented as a sequence \(x[n]\), where \(n\) is an integer. The conversion from continuous to discrete is achieved by sampling.
1.2.2 Deterministic vs. Random
Deterministic signals can be described by a mathematical formula or a known pattern (e.g., a sinusoid). Random signals are unpredictable and modeled using probability theory (e.g., noise or speech). Many real-world signals are treated as stochastic processes.
1.2.3 Periodic vs. Aperiodic
A periodic signal repeats itself exactly after a fixed period \(T\) (continuous) or \(N\) (discrete): \(x(t+T)=x(t)\). An aperiodic signal does not exhibit such repetition. Periodic signals are analyzed using Fourier series, while aperiodic signals require the Fourier transform.
1.3 Basic Signal Operations
Fundamental operations modify signals in time or amplitude.
1.3.1 Scaling, Shifting, and Reversal
- Amplitude scaling: multiplying a signal by a constant (e.g., \(y(t)=a x(t)\)).
- Time shifting: delaying or advancing a signal in time (e.g., \(y(t)=x(t-t_0)\)).
- Time reversal: reflecting the signal about \(t=0\) (e.g., \(y(t)=x(-t)\)).
1.3.2 Convolution and Correlation
Convolution combines two signals to produce a third, representing the output of a linear time-invariant system. For continuous signals: \((x*h)(t)=\int_{-\infty}^{\infty} x(\tau)h(t-\tau)d\tau\). Correlation measures similarity between signals as a function of time shift, used in pattern recognition and synchronization.
2 Mathematical Foundations
2.1 Fourier Analysis
Fourier analysis decomposes signals into sinusoidal components, revealing their frequency content.
2.1.1 Fourier Series
Any periodic signal satisfying Dirichlet conditions can be expressed as a sum of complex exponentials: \(x(t)=\sum_{k=-\infty}^{\infty} c_k e^{j2\pi k f_0 t}\), where \(f_0=1/T\) and \(c_k\) are the Fourier coefficients.
2.1.2 Fourier Transform (Continuous and Discrete)
The continuous-time Fourier transform maps a signal \(x(t)\) to its frequency-domain representation \(X(f)=\int_{-\infty}^{\infty} x(t)e^{-j2\pi f t}dt\). The discrete-time Fourier transform (DTFT) applies to sequences: \(X(e^{j\omega})=\sum_{n=-\infty}^{\infty} x[n]e^{-j\omega n}\). The discrete Fourier transform (DFT) is a sampled version used for finite-length sequences.
2.1.3 Fast Fourier Transform (FFT)
The FFT is an efficient algorithm to compute the DFT, reducing complexity from \(O(N^2)\) to \(O(N\log N)\). It revolutionized digital signal processing by enabling real-time spectral analysis and convolution.
2.2 Laplace and Z-Transforms
These transforms extend Fourier analysis to more general signals and systems, particularly for stability and transient analysis.
2.2.1 Laplace Transform for Analog Signals
The Laplace transform \(X(s)=\int_{0}^{\infty} x(t)e^{-st}dt\) (with \(s=\sigma+j\omega\)) is used for continuous-time linear systems. It converts differential equations into algebraic equations and is essential for analyzing poles and zeros.
2.2.2 Z-Transform for Digital Signals
The Z-transform \(X(z)=\sum_{n=-\infty}^{\infty} x[n]z^{-n}\) generalizes the DTFT. It is the primary tool for discrete-time system analysis, enabling the representation of difference equations and transfer functions.
2.2.3 Region of Convergence
Both Laplace and Z-transforms are defined only for values of \(s\) or \(z\) where the integral or sum converges—the region of convergence (ROC). The ROC determines causality and stability of the corresponding system.
2.3 Linear Time-Invariant (LTI) Systems
LTI systems obey superposition and time-invariance, making them amenable to transform-based analysis.
2.3.1 Impulse Response and Transfer Function
An LTI system is fully characterized by its impulse response \(h(t)\) or \(h[n]\). The output is the convolution of input and impulse response. The transfer function is the Laplace or Z-transform of the impulse response, representing the system’s frequency response.
2.3.2 Stability and Causality
A system is stable if bounded input yields bounded output (BIBO). For LTI systems, stability requires the impulse response to be absolutely integrable (continuous) or absolutely summable (discrete) and the ROC to include the imaginary axis (s-plane) or unit circle (z-plane). Causality implies that output depends only on past and present inputs.
3 Analog Signal Processing
3.1 Passive and Active Filters
Filters selectively pass or attenuate frequency components. Passive filters use resistors, capacitors, and inductors; active filters incorporate operational amplifiers for gain and isolation.
3.1.1 Low-Pass, High-Pass, Band-Pass, Band-Stop
- Low-pass: passes frequencies below a cutoff.
- High-pass: passes frequencies above a cutoff.
- Band-pass: passes a range of frequencies.
- Band-stop (notch): attenuates a specific range.
3.1.2 Butterworth, Chebyshev, Bessel Designs
- Butterworth: maximally flat passband, gradual roll-off.
- Chebyshev: steeper roll-off with passband ripple.
- Bessel: linear phase response, preserving waveform shape.
3.2 Modulation and Demodulation
Modulation encodes information onto a carrier wave; demodulation recovers it.
3.2.1 Amplitude Modulation (AM)
The carrier amplitude varies proportionally to the message signal. AM is simple but susceptible to noise. Demodulation uses envelope detection or synchronous detection.
3.2.2 Frequency Modulation (FM)
The carrier frequency varies with the message amplitude. FM offers better noise immunity than AM but requires wider bandwidth.
3.3 Operational Amplifier Circuits
Operational amplifiers (op-amps) are building blocks for analog signal processing. Key circuits include inverting/non-inverting amplifiers, integrators, differentiators, and active filters. Op-amps provide high gain, high input impedance, and low output impedance.
4 Digital Signal Processing
4.1 Sampling and Quantization
Converting an analog signal to digital involves sampling in time and quantizing in amplitude.
4.1.1 Nyquist-Shannon Sampling Theorem
A bandlimited signal with maximum frequency \(f_{\text{max}}\) can be perfectly reconstructed from samples taken at a rate \(f_s \geq 2f_{\text{max}}\). This minimum rate is the Nyquist rate.
4.1.2 Aliasing and Anti-Aliasing Filters
If sampling is below the Nyquist rate, high-frequency components appear as lower-frequency artifacts (aliasing). An anti-aliasing low-pass filter is applied before sampling to remove frequencies above half the sampling rate.
4.1.3 Quantization Noise
Quantization maps continuous amplitudes to discrete levels, introducing error. The quantization noise power depends on the number of bits; more bits reduce noise.
4.2 Digital Filters
Digital filters process discrete-time signals using arithmetic operations.
4.2.1 Finite Impulse Response (FIR) Filters
FIR filters have an impulse response of finite duration and are inherently stable.
4.2.1.1 Linear Phase Characteristics
A symmetric FIR filter exhibits linear phase, meaning all frequency components experience the same delay, preserving waveform shape.
4.2.1.2 Window Method Design
The ideal infinite impulse response is truncated using a window function (e.g., Hamming, Blackman) to obtain a practical FIR filter. The choice of window trades sidelobe suppression against main lobe width.
4.2.2 Infinite Impulse Response (IIR) Filters
IIR filters have feedback, allowing lower order for sharp transitions but may be unstable.
4.2.2.1 Bilinear Transform
Maps the analog \(s\)-plane to the digital \(z\)-plane via \(s = \frac{2}{T}\frac{1-z^{-1}}{1+z^{-1}}\). It preserves stability and avoids aliasing but warps the frequency axis.
4.2.2.2 Impulse Invariance Method
Samples the analog impulse response to obtain digital filter coefficients. It preserves the time-domain shape but can cause aliasing if the analog filter is not bandlimited.
4.3 Multirate Signal Processing
Multirate systems operate with multiple sampling rates.
4.3.1 Decimation and Interpolation
Decimation reduces the sampling rate by discarding samples (after anti-aliasing filtering). Interpolation increases the sampling rate by inserting zeros and then low-pass filtering.
4.3.2 Polyphase Decomposition
A filter’s impulse response is split into multiple phases, each associated with a decimated or interpolated sequence. Polyphase structures reduce computational complexity in multirate implementations.
5 Advanced Techniques and Applications
5.1 Statistical Signal Processing
Statistical methods handle signals corrupted by noise or randomness.
5.1.1 Estimation Theory (MMSE, MAP)
Estimation aims to recover unknown parameters from noisy observations. Minimum mean square error (MMSE) estimation minimizes the expected squared error. Maximum a posteriori (MAP) estimation incorporates prior knowledge.
5.1.2 Wiener and Kalman Filtering
Wiener filtering provides optimal linear MMSE estimation for stationary signals. The Kalman filter extends this to non-stationary, dynamic systems, recursively updating state estimates using a predictor-corrector framework.
5.2 Adaptive Filters
Adaptive filters adjust their coefficients automatically based on the input signal, useful in unknown or time-varying environments.
5.2.1 Least Mean Squares (LMS) Algorithm
LMS updates coefficients in the direction opposite to the instantaneous gradient of the mean squared error. It is simple and robust, with a trade-off between convergence speed and steady-state error.
5.2.2 Recursive Least Squares (RLS)
RLS minimizes a weighted least squares criterion, offering faster convergence than LMS at the expense of higher computational cost.
5.3 Spectral Estimation and Time-Frequency Analysis
These methods analyze how the frequency content of a signal changes over time.
5.3.1 Short-Time Fourier Transform (STFT)
The STFT divides a signal into overlapping segments and computes the Fourier transform of each, producing a spectrogram. It provides a time-frequency representation with fixed resolution (determined by window length).
5.3.2 Wavelet Transforms
Wavelet transforms use scaled and shifted basis functions (wavelets) to achieve variable time-frequency resolution: fine time resolution at high frequencies and fine frequency resolution at low frequencies. The discrete wavelet transform (DWT) is widely used in compression and denoising.
5.4 Compression and Coding
Signal compression reduces data size for storage or transmission.
5.4.1 Lossless vs. Lossy Compression
Lossless compression reconstructs the original signal exactly (e.g., ZIP, FLAC). Lossy compression discards perceptually irrelevant information for higher compression ratios (e.g., MP3, JPEG).
5.4.2 MPEG, JPEG, and Speech Codecs
- MPEG: video compression standards using motion compensation and DCT.
- JPEG: still-image compression based on DCT and quantization.
- Speech codecs: e.g., CELP, AMR, designed for low bitrate voice communication.
5.5 Biomedical Signal Processing
Biomedical signals capture physiological activity.
5.5.1 Electrocardiography (ECG) Analysis
ECG records the heart’s electrical activity. Signal processing techniques filter noise, detect QRS complexes, and analyze heart rate variability for diagnosis.
5.5.2 Electroencephalography (EEG) Processing
EEG measures brain electrical signals. Techniques include artifact removal, spectral analysis for brain rhythms (alpha, beta, etc.), and event-related potential detection.
5.6 Audio and Image Processing
These fields apply signal processing to human-perceptible media.
5.6.1 Noise Reduction and Echo Cancellation
Noise reduction uses spectral subtraction or adaptive filtering to suppress background noise. Echo cancellation in telephony employs adaptive filters to remove the far-end signal from the near-end microphone.
5.6.2 Image Enhancement and Restoration
Enhancement techniques (histogram equalization, sharpening) improve visual quality. Restoration algorithms (deconvolution, inpainting) reverse known degradations such as blur or noise.