Recent advances in high throughput imaging have led to the creation of massive image repositories, where human analysis is often infeasible. Automated image analysis offers a promising alternative for reducing analysis time by several orders of magnitude. In order to design algorithms that are robust and practically usable, there are a variety of design considerations that require investigation.

This dissertation explores three specific considerations in visual segmentation and detection, namely domain specific priors, scalability, and semantics inherent in the data. The first part of this work proposes a framework that adapts a generic segmentation/tracing technique to application specific ones using priors such as topological dynamics and shape in a Markov Random Field (MRF) setting. Subsequently, techniques to scale algorithms for tracing a large number of targets are explored. These tracing algorithms are based on graph diffusion, and are capable of scaling gracefully with increasing number of targets. The final part of this work explores semantic attributes that humans utilize for object detection in weakly supervised settings. Kernel methods are utilized to learn classifiers in multiple feature spaces proposed in this work for detecting non-rigid objects.

This work adopts the problem of connectomics (neuronal circuit reconstruction from Electron Micrographs) to illustrate the applicability of proposed techniques. Specifically, the segmentation and tracing algorithms are shown to isolate neuronal structures in 3D while the detection algorithms localize synaptic junctions, thus taking a step closer to automated neural circuit constructions from raw image data. Further, the proposed algorithms are also applied on natural image and video data to illustrate their generalization capability.