17-《ARKit by Tutorials》读书笔记4：面部识别AR

ARKit系列文章目录

本文是Ray Wenderlich上《ARKit by Tutorials》的读书笔记,主要讲内容概要和读后感

没错,本文主要讲iPhone X的前置TrueDepth摄像头的AR效果!主要功能:

• 面部检测和追踪
• 实时面部表情追踪
• 同时捕捉彩色图像,和深度图像
• 光照估计

初始化配置

面部AR的启动和其他的差不多,只需要将配置项换成 ARFaceTrackingConfiguration() 就可以了,错误处理也类似:

func resetTracking() {
  // 1
  guard ARFaceTrackingConfiguration.isSupported else {
    updateMessage(text: "Face Tracking Not Supported.")
    return
  }

  // 2
  updateMessage(text: "Looking for a face.")

  // 3
  let configuration = ARFaceTrackingConfiguration()
  configuration.isLightEstimationEnabled = true /* default setting */
  configuration.providesAudioData = false /* default setting */

  // 4
  session.run(configuration, options: 
              [.resetTracking, .removeExistingAnchors])
}
复制代码

func session(_ session: ARSession, didFailWithError error: Error) {
  print("** didFailWithError")
  updateMessage(text: "Session failed.")
}

func sessionWasInterrupted(_ session: ARSession) {
  print("** sessionWasInterrupted")
  updateMessage(text: "Session interrupted.")
}

func sessionInterruptionEnded(_ session: ARSession) {
  print("** sessionInterruptionEnded")
  updateMessage(text: "Session interruption ended.")
}
复制代码

人脸追踪

ARKit在检测到人脸时,会添加一个 ARFaceAnchor 到场景中,我们就可以用这个锚点来实现定位和追踪功能.

如果有两张人脸,ARKit只会追踪最大,最具有辨识度的那张脸.同时会在人脸上添加一个人脸坐标系:

来持有几何体:

func createFaceGeometry() {
  updateMessage(text: "Creating face geometry.")

  let device = sceneView.device!

  let maskGeometry = ARSCNFaceGeometry(device: device)!
  mask = Mask(geometry: maskGeometry)
}
复制代码

这个方法可以在 viewDidLoad 里面直接调用,无需等到人脸出现再创建.

最后将 mask 添加到视图上以显示出来:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
  anchorNode = node
  //将mask显示出来
  setupFaceNodeContent()
}

func setupFaceNodeContent() {
  guard let node = anchorNode else { return }

  node.childNodes.forEach { $0.removeFromParentNode() }

  if let content = mask {
    node.addChildNode(content)
  }
}
复制代码

前面创建的几何体 ARSCNFaceGeometry 因为还没有识别到人脸,可能是空的.所以我们需要在后面的update方法中更新.

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
  guard let faceAnchor = anchor as? ARFaceAnchor else { return }
  updateMessage(text: "Tracking your face.")
  // 根据锚点更新几何体
  mask?.update(withFaceAnchor: faceAnchor)
}
// Tag: ARFaceAnchor Update
func update(withFaceAnchor anchor: ARFaceAnchor) {
  let faceGeometry = geometry as! ARSCNFaceGeometry
  faceGeometry.update(from: anchor.geometry)
}
复制代码

默认情况下灯光设置是这样:

/* default settings */
sceneView.automaticallyUpdatesLighting = true 
sceneView.autoenablesDefaultLighting = false 
sceneView.scene.lightingEnvironment.intensity = 1.0 
复制代码

但我们也可以根据环境调整,以实现不同的效果:

func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
  // 1
  guard let estimate = session.currentFrame?.lightEstimate else {
    return
  }

  // 2 在ARKit中1000意味着光强为中等
  let intensity = estimate.ambientIntensity / 1000.0
  sceneView.scene.lightingEnvironment.intensity = intensity

  // 3
  let intensityStr = String(format: "%.2f", intensity)
  let sceneLighting = String(format: "%.2f",
    sceneView.scene.lightingEnvironment.intensity)

  // 4
  print("Intensity: \(intensityStr) - \(sceneLighting)")
}
复制代码

blend shapes表情动画

如何实现Animojis中的表情动画效果呢?这就用到了blend shapes,它本质是一个字典,key是 ARFaceAnchor.BlendShapeLocation 常量,而value则是浮点数,范围0.0(自然状态)~1.0(最大移动状态).

比如,我们想添加一个眨眼的效果,取出 eyeBlinkLeft :

这个字典的是随着ARFaceAnchor更新而更新的:

// - Tag: ARFaceAnchor Update
func update(withFaceAnchor anchor: ARFaceAnchor) {
  blendShapes = anchor.blendShapes
}

// - Tag: BlendShapeAnimation
var blendShapes: [ARFaceAnchor.BlendShapeLocation: Any] = [:] {
  didSet {
    guard
      // Brow
      let browInnerUp = blendShapes[.browInnerUp] as? Float
      // Right eye
let eyeLookInRight = blendShapes[.eyeLookInRight] as? Float,
let eyeLookOutRight = blendShapes[.eyeLookOutRight] as? Float,
let eyeLookUpRight = blendShapes[.eyeLookUpRight] as? Float,
let eyeLookDownRight = blendShapes[.eyeLookDownRight] as? Float,
let eyeBlinkRight = blendShapes[.eyeBlinkRight] as? Float
      else { return }

    // 在处理动画
}
复制代码

实现后的效果

ReplayKit录像

ReplayKit是在iOS 9引入的,用来录制音频,视频和麦克风,主要有两个类:

• RPScreenRecorder :用来启动/停止录制的单例.
• RPPreviewViewController :录制完成后的预览控制器.

:

• iOS屏幕录制和广播
• 广播配对
• 快速摄像头切换
• 应用内屏幕捕捉

开始录制

主要代码

let sharedRecorder = RPScreenRecorder.shared()
private var isRecording = false 
复制代码

// Private functions
private func startRecording() {
  // 1
  self.sharedRecorder.isMicrophoneEnabled = true

  // 2
  sharedRecorder.startRecording( handler: { error in
    guard error == nil else {
      print("There was an error starting the recording: \(String(describing: error?.localizedDescription))")
      return
    }

    // 3
    print("Started Recording Successfully")
    self.isRecording = true

    // 4
    DispatchQueue.main.async {
      self.recordButton.setTitle("[ STOP RECORDING ]", for: .normal)
      self.recordButton.backgroundColor = UIColor.red
    }
  })
} 
复制代码

停止录制

主要代码

func stopRecording() {
  // 1
  self.sharedRecorder.isMicrophoneEnabled = false

  // 2
  sharedRecorder.stopRecording( handler: { 
    previewViewController, error in
    guard error == nil else {
      print("There was an error stopping the recording: \(String(describing: error?.localizedDescription))")
      return
    }

    // 3
    if let unwrappedPreview = previewViewController {
      unwrappedPreview.previewControllerDelegate = self
      self.present(unwrappedPreview, animated: true, completion: {})
    }
  })

  // 4
  self.isRecording = false
  DispatchQueue.main.async {
    self.recordButton.setTitle("[ RECORD ]", for: .normal)
    self.recordButton.backgroundColor = UIColor(red: 0.0039, 
      green: 0.5882, blue: 1, alpha: 1.0) /* #0196ff */
  }
}
复制代码

效果如图:

另外,还可以实现 sharedRecorder 的代理方法,监听错误回调和状态改变:

// RPScreenRecorderDelegate methods
func screenRecorder(_ screenRecorder: RPScreenRecorder, 
  didStopRecordingWith previewViewController: RPPreviewViewController?,
  error: Error?) {
  guard error == nil else {
    print("There was an error recording: \(String(describing:
      error?.localizedDescription))")
    self.isRecording = false
    return
  }
}

func screenRecorderDidChangeAvailability(_ screenRecorder:  
  RPScreenRecorder) {
  recordButton.isEnabled = sharedRecorder.isAvailable
  if !recordButton.isEnabled {
    self.isRecording = false
  }
}
复制代码

使用方法不算太难,此处不再过多展开,详情可购买阅读正版书籍.

第四部分读书笔记结束!