Filesystem Management with Flysystem at PHP UK 2023
1 like141 views
The document provides an overview of Flysystem, a PHP library for managing file systems through a unified API, addressing issues like vendor lock-in and inconsistencies across different storage solutions. It covers implementation strategies, including exceptions handling, wrapper patterns, and testing methodologies, to facilitate efficient file operations across various storage services like AWS S3 and Azure. Additionally, it emphasizes the importance of understanding the Flysystem API and its adapters for enhanced file management and integration with PHP frameworks.
![Flysystem API
• Write
- write(
string $location,
string $contents,
array $config = []
): void
- writeStream(
string $location,
$contents,
array $config = []
): void](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-33-320.jpg)
![Flysystem API
• Directory
- createDirectory(
string $location,
array $config = []
): void
- listContents(
string $location,
bool $deep = self::LIST_SHALLOW
): DirectoryListing](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-36-320.jpg)
![Flysystem API
• Files
- move(
string $source,
string $destination,
array $config = []
): void
- copy(
string $source,
string $destination,
array $config = []
): void](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-37-320.jpg)
![Flysystem API
• Metadata
- lastModified(string $path): int
- fileSize(string $path): int
- mimeType(string $path): string
- checksum(
string $path,
array $config = []
): string](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-38-320.jpg)
![Flysystem API
• URLs
- S3, Azure, Google Cloud, WebDAV
‣ publicUrl(
string $path,
array $config = []
): string
‣ temporaryUrl(
string $path,
DateTimeInterface $expiresAt,
array $config = []
): string](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-40-320.jpg)
![Implementation Strategies
• Con
fi
guration
- Injectable con
fi
guration for adapters
'filesystem' => [
'local' => [
'directory' => '/tmp/filesystem/files/',
],
'awss3' => [
'client' => [
'credentials' => [
'key' => 'key...',
'secret' => 'secret...',
],
'region' => 'us‑west‑1',
'version' => 'latest',
],
'bucket' => 'my-bucket-name',
],
],](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-45-320.jpg)
![Implementation Strategies
class FileManagerFactory
{
public function __construct(
protected FileStorageAdapterFactory $adapterFactory
) {}
public function create(array $mounts): FileManager
{
$mountManageConfig = [];
foreach ($mounts as $storage => $config) {
$mountManageConfig[$storage] = new Filesystem(
$this->adapterFactory->create($storage, $config)
);
}
return new FileManager(new MountManager($mountManageConfig));
}](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-61-320.jpg)
![Implementation Strategies
class FileStorageAdapterFactory
{
public function create(string $adapter, array $config): FilesystemAdapter
{
return match ($adapter) {
'awss3' => new AwsS3V3Adapter($config['client'], $config['bucket']),
'local' => new LocalFilesystemAdapter($config['directory']),
};
}](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-62-320.jpg)
![Testing
public function testRead(): void
{
$text = 'Test text!';
$testPath = uniqid('/tmp/test/') . '.txt';
$filesystemMock = $this->getMockBuilder(Filesystem::class)
->disableOriginalConstructor()
->onlyMethods(['read'])
->getMock();
$filesystemMock->method('read')
->with($testPath)
->willReturn($text);
$service = new FileStorageService($filesystemMock);
$contents = $service->read($testPath);
$this->assertSame($text, $contents);
}](https://image.slidesharecdn.com/filesystemmanagementwithflysystemphpuk2023-230216135600-2cca98c7/85/Filesystem-Management-with-Flysystem-at-PHP-UK-2023-68-320.jpg)
Filesystem Management with Flysystem at PHP UK 2023
- 1. Filesystem Management with Flysystem Mark Niebergall
- 2. 👏 Thank you! • Organisers • Delegates • Speakers • Sponsors - Nucleus Security
- 3. 🧑💻 PHP Community • You are amazing • Attending a conference - Learn - Network - Have fun - Apply what you learned
- 4. 🧑💻 PHP Community • You made it to Thursday afternoon! 🤯
- 5. • 🙋 Poll: Who is already using Flysystem?
- 8. 🤔 Question • How do you e ff ectively manage fi les with code?
- 9. 🤔 Question • Common solution - fopen - fi le_get_contents - fi le_put_contents - delete - chmod
- 11. AwsS3Service Local In Memory AWS S3 Azure SFTP File LocalService InMemoryService AzureService SftpService 😵💫 The Problem
- 12. • “Flysystem is a fi le storage library for PHP. It provides one interface to interact with many di ff erent types of fi lesystems. When you use Flysystem, you’re not only protected from vendor lock-in, you’ll also have a consistent experience for which ever storage is right for you.”
- 14. ✅ Objective • Familiarity with Flysystem API • Ideas on implementation • Improved fi lesystem interactions
- 15. 📋 Overview • Adapters • Flysystem API • Implementation Strategies • Testing
- 16. Adapters
- 18. Adapters • Traditional - Di ff erent code for local vs external - Di ff erent code for each adapter
- 19. Adapters • Use case - Local for development - SFTP for QA - S3 storage for Production
- 20. Adapters • Use case - Variety of fi le types
- 21. Adapters public function readLocal(string $location): string { return file_get_contents($location); } public function readS3(string $location): string { // read file from S3 storage return 'File Contents'; } public function getContents(string $storage, string $location): string { return match ($storage) { 'local' => $this->readLocal($location), 'awss3' => $this->readS3($location), default => throw new InvalidArgumentException('Not configured'), }; }
- 22. Adapters • Flysystem is con fi gurable - Local - External services
- 23. Adapters • O ffi cial Adapters - Local - Local - InMemory
- 24. Adapters • O ffi cial Adapters - External - AWS S3 - AsyncAws S3 - Azure Blob Storage - Google Cloud Storage - SFTP - WebDAV
- 25. Adapters • Community Adapters - GitLab
- 26. Adapters • Decorators - Read-Only ‣ $adapter = new InMemoryFilesystemAdapter(); $readOnlyAdapter = new LeagueFlysystemReadOnlyReadOnlyFile systemAdapter($adapter); $filesystem = new Filesystem($adapter);
- 27. Adapters • Decorators - Path Pre fi xing ‣ $pathPrefixedAdapter = new LeagueFlysystemPathPrefixngPathPref ixedAdapter($adapter, 'a/path/prefix');
- 28. Flysystem API
- 30. Flysystem API • Decent documentation online - https:// fl ysystem.thephpleague.com/docs/ - Make sure you are on the correction version of documentation!
- 31. Flysystem API • Exists - fileExists(string $location): bool - directoryExists(string $location): bool - has(string $location): bool ‣ $this->adapter->fileExists( $path ) || $this->adapter->directoryExists( $path );
- 32. Flysystem API • File CRUD - C: Write - R: Read - U: Move - D: Delete
- 33. Flysystem API • Write - write( string $location, string $contents, array $config = [] ): void - writeStream( string $location, $contents, array $config = [] ): void
- 34. Flysystem API • Read - read(string $location): string - readStream(string $location)
- 35. Flysystem API • Delete - delete(string $location): void - deleteDirectory(string $location): void
- 36. Flysystem API • Directory - createDirectory( string $location, array $config = [] ): void - listContents( string $location, bool $deep = self::LIST_SHALLOW ): DirectoryListing
- 37. Flysystem API • Files - move( string $source, string $destination, array $config = [] ): void - copy( string $source, string $destination, array $config = [] ): void
- 38. Flysystem API • Metadata - lastModified(string $path): int - fileSize(string $path): int - mimeType(string $path): string - checksum( string $path, array $config = [] ): string
- 39. Flysystem API • Permissions - setVisibility( string $path, string $visibility ): void - visibility(string $path): string
- 40. Flysystem API • URLs - S3, Azure, Google Cloud, WebDAV ‣ publicUrl( string $path, array $config = [] ): string ‣ temporaryUrl( string $path, DateTimeInterface $expiresAt, array $config = [] ): string
- 41. Flysystem API • Frameworks - Bundled with Laravel - Available with Symfony - Can be used with other frameworks too
- 44. Implementation Strategies • Con fi guration • Exception Handling • Wrapper • Mount Manager
- 45. Implementation Strategies • Con fi guration - Injectable con fi guration for adapters 'filesystem' => [ 'local' => [ 'directory' => '/tmp/filesystem/files/', ], 'awss3' => [ 'client' => [ 'credentials' => [ 'key' => 'key...', 'secret' => 'secret...', ], 'region' => 'us‑west‑1', 'version' => 'latest', ], 'bucket' => 'my-bucket-name', ], ],
- 46. Implementation Strategies • Con fi guration - Application fl exibility - Variety of sources ‣ Database ‣ PHP con fi guration fi les
- 47. Implementation Strategies • Con fi guration - UI interface to con fi gure - Scalability
- 48. Implementation Strategies • Exception Handling - Generic exception ‣ FilesystemException - Speci fi c exceptions ‣ UnableToReadFile ‣ UnableToWriteFile ‣ UnableToDeleteFile ‣ UnableToCheckExistence ‣ ...
- 49. Implementation Strategies • Exception Handling - Full list of exceptions ‣ https:// fl ysystem.thephpleague.com/docs/usage/ fi lesystem-api/
- 50. Implementation Strategies • Exception Handling 1. Catch Flysystem speci fi c exceptions 2. Handle the exception 3. Throw own meaningful exception
- 51. Implementation Strategies public function read(string $location): string { try { $contents = $this->filesystem->read($location); } catch (UnableToReadFile $exception) { // handle unable to read exception // throw an exception } catch (FilesystemException $exception) { // handle general error // throw an exception } return $contents; }
- 52. Implementation Strategies • Wrapper - Wrapper or Facade Pattern - Abstracts away underlying package - Swappable package - Unit testable
- 53. Implementation Strategies • Wrapper - Useful pattern for many services ‣ Filesystem ‣ Email ‣ SMS ‣ Database ‣ Queue system
- 54. Implementation Strategies class FileStorageService { public function __construct( protected Filesystem $filesystem ) {} public function read(string $location): string { return $this->filesystem->read($location); }
- 55. Implementation Strategies • Mount Manager - Scenario ‣ Retrieve fi les from an external service ‣ Write locally for processing
- 56. Implementation Strategies • Mount Manager - Scenario ‣ Move fi les between external adapters upon an event
- 57. Implementation Strategies • Mount Manager - Scenario ‣ Many customers ‣ Reports created, stored locally ‣ Reports delivered based on con fi guration ‣ Variety of fi le storage locations
- 58. Implementation Strategies • Mount Manager - Interact with fi les in many storage services - Specify adapter in location ‣ awss3://subfolder/ fi le.txt ‣ local://path/to/ fi le.txt
- 60. Implementation Strategies class FileManager { public function __construct( protected MountManager $mountManager ) {} public function read(string $location): string { return $this->mountManager->read($location); }
- 61. Implementation Strategies class FileManagerFactory { public function __construct( protected FileStorageAdapterFactory $adapterFactory ) {} public function create(array $mounts): FileManager { $mountManageConfig = []; foreach ($mounts as $storage => $config) { $mountManageConfig[$storage] = new Filesystem( $this->adapterFactory->create($storage, $config) ); } return new FileManager(new MountManager($mountManageConfig)); }
- 62. Implementation Strategies class FileStorageAdapterFactory { public function create(string $adapter, array $config): FilesystemAdapter { return match ($adapter) { 'awss3' => new AwsS3V3Adapter($config['client'], $config['bucket']), 'local' => new LocalFilesystemAdapter($config['directory']), }; }
- 63. Testing
- 65. Testing • Historically problematic testing fi le interactions - Test fi les - Inject fi le content - git ignore directory for testing fi les - Skip testing that code - Integration tests vs Unit tests
- 66. Testing public function testRead(): void { $file = __DIR__ . '/TestingFile.txt'; $service = new FileStorageLocalLegacyService(); $contents = $service->read($file); $this->assertSame( 'Test file contents 123 ...', $contents ); }
- 67. Testing • Flysystem abstraction layer allows for testing - Mock calls to fi le interactions - Pushes for single purpose code - Centralized fi lesystem management
- 68. Testing public function testRead(): void { $text = 'Test text!'; $testPath = uniqid('/tmp/test/') . '.txt'; $filesystemMock = $this->getMockBuilder(Filesystem::class) ->disableOriginalConstructor() ->onlyMethods(['read']) ->getMock(); $filesystemMock->method('read') ->with($testPath) ->willReturn($text); $service = new FileStorageService($filesystemMock); $contents = $service->read($testPath); $this->assertSame($text, $contents); }
- 69. 🗣 Discussion
- 70. 🗣 Discussion • Bene fi ts of Flysystem? • Cons of Flysystem?
- 71. 🗣 Discussion • Security • Credentials
- 72. 🗣 Discussion • Conversion cost • Scalability • Maintenance
- 73. 📋 Review • External Services • Flysystem API • Implementation Strategies • Testing
- 74. Mark Niebergall @mbniebergall • PHP since 2005 • Masters degree in MIS • Senior Software Engineer • Vulnerability Management project (security scans) • Utah PHP Co-Organizer • CSSLP, SSCP Certi fi ed and Exam Developer • Endurance sports, outdoors
- 77. Filesystem Management with Flysystem • Questions? • I’ll stay after to answer more questions, help with setup and con fi gs
- 78. 👀 References • https:// fl ysystem.thephpleague.com/docs/usage/ fi lesystem-api/ • https://github.com/thephpleague/ fl ysystem