Learning Sensor Multiplexing Design through Back-propagation

摘要: Recent progress on many imaging and vision tasks has been driven by the use of deep feed-forward neural networks, which are trained propagating gradients a loss defined final output, back through network up to first layer that operates directly image. We propose back-propagating one step further---to learn camera sensor designs jointly with networks carry out inference images they capture. In this paper, we specifically consider design problems in typical color camera---where is able measure only channel at each pixel location, computational required reconstruct full sensor's multiplexing pattern encoding it as whose learnable weights determine channel, from among fixed set, will be measured location. These those reconstruction corresponding measurements produce Our achieves significant improvements accuracy over traditional Bayer used most cameras. It automatically learns employ sparse measurement approach similar recent design, moreover, improves upon learning an optimal layout for these measurements.